While there are many reasons for this, and everyone not flying in AvA (like me for example) has his own weighted combination of them, I'd guess one of the major ones you have stated yourself: "If only numbers were up par". Numbers generate numbers. Once an arena goes below a certain level, a vicious circle starts. Smaller numbers on login screen will result in even less number of players logging in. If all those players not flying AvA just because of low numbers would log in - the numbers might be actually surprisingly high 
Very true. I think you see it now with the early- and mid-war arenas as well. No one wants to be the first. The odd thing is that most people who fly there will tell you that AvA and mid-war arenas have some of the very highest quality fights. You really don't need 100 people to have a great place to fly.
I think the biggest two problems with plane-limited arenas are (a) too many people refuse to fly in an arena where their favorite plane is not enabled; and (b) too many squadrons are wedded to one plane type. To make it worse, typically these planes will be 1945 fighters. The result, over time, is that the vast majority of people will gravitate to an arena that has 1945 fighters.
Parenthetically, my memory disagrees with Lusche's. When HTC first enabled the arena split there were no early- or mid-war planes enabled in the late war arenas, which made sense. HTC gave in too quickly, in my opinion, to the people who still wanted to show off by shooting down Spit 16s in Spit Is. The exodus from early- and mid-war arenas began then, and neither has ever recovered. It was the one thing about the arena split that I thought was a mistake.
- oldman