Consider this. You're a tanker and have wreckage tuned on. There's a Panzer hulk between you and some enemies so you sneak in behind it to camoflauge yourself. I'm one of the enemies. I have wreckage turned off and what I see is you sitting in the open.
There's no doubt it would affect frame rate just as skid marks do on the track in a racing game but here even the lowest end computer will be forced to run these extra textures just to keep things even and they'd have to be there for everyone, whether you witnessed it or not.
-1