To destroy a building(s) with these two types of ammunition ie .50cal and 20mm, is not realistic. They would make holes for a fact, but one plane on one pass could not destroy one of these structures. I have done it myself, but it's not right. My wish is to make them harder to destroy with machine guns and 20mm cannon if at all.
A group of planes would probably tear down a building eventually, but not a single plane.