Nobody ever claimed that the J bodies were fast, at least in stock form. All cars, from all vendors, of that period were slow, due to a number of factors. Look at what a 1980 Z28 or a 1981 Trans Am, or a 1980 Mustang as examples. There were some years in which the fastest accelerating vehicle (0-60) was the L'il Red Express truck, faster than the Corvette even. As slow as the J bodies were, the previous few years of the late B bodies were slower; in addition to having the same wheezy, low compression, smogger engines, they were carrying a lot more weight around (whatever you spoke about regarding adding people, gas, and your dog would add the same amount of weight to a late B body), but the late B bodies are starting with a 500-600 lbs. weight penalty. Only thing the late B Bodies had were the 360 4 BBL, and a better/stronger rear end in some cases. By today's standards, even a 68 383 Charger, factory original, is also a slug. You can't use today's standards and apply them to the cars of the past.
Fast forward to today. You can put whatever you want into a J body. From a 1 BBL /6 to a Big Block to a Gen III Hemi. Whatever you put into the engine compartment, you are still pushing a significantly lower amount of weight around compared to a late B Bodies. The B Bodies do have a better front suspension set up in theory, but for all intents and purposes, it doesn't make much of a difference for a daily driven car. As for road racers, you probably shouldn't be using either a late B Body or a J Body, even though there are suspension kits available for both cars.