There is no such thing as collateral damage. You don't think the armed forces know their weapons and munitions? When a weapon is used, it is intentional. Any resulting damage is calculated and intentional. Perhaps not wanted, but still intentional and a calculated known factor in the use of weapons.
I think the term was invented by someone senior who didn't want to take responsibility for what they were doing.
I guess in the grand scheme of things then your right there really is no collateral damage, but the term generally refers to situations like, we were aiming at that bunker there and the school that was next door sustained some damage as well, now when they were going after the bunker they weren't intending to damage the school as well, thus making the damage the school sustained classified as collateral damage, or when you go after a target and something goes wrong and a different non military target gets hit instead, that's collateral damage, but yes these things are ultimately unavoidable and thus going to happen no matter what, all we can do is try and make weapons that are smarter & more accurate to make the chances of collateral damage happening less & less likely.
When it comes down to it your going to have the need for a term to describe what happens when a non military target takes damage, and collateral damage fits that role fine, otherwise what would you have us call it, a whoopsie?