I vaguely recall that HT once posted a possible solution to this. Something on the lines of, "your ordnance only does damage if you are still alive 5 seconds after it hits".
That was talked about in the context of the bomb-and-bailers, but it is unfair to those who are shot down after releasing their ords. However, we do already have the disappearing ords to prevent the spawn PT-launch torps-bail-repeat that used to happen.
As to the OP, this wish is absolutely silly for the same reason, and is a thinly veiled screen for a "you sank my cv" whine. Glad you were having fun furballing, but since the CV is sunk, go find (or start) a furball somewhere else -- problem solved.