Depends. Are the costs of any of the following prohibitive for a middle class person?
1. Boat
2. RV
3. Vacation
Just take the money from all three of those common middle class pursuits, put it together, and you have aviation!

It's all about priorities and decisions. You can get into aviation for the price of a modest new car. I saw a really nice looking TriPacer in Prineville this morning selling for $17k, for instance. As for ongoing costs, fuel is the biggest, most immediate cost. 4-9 gallons per hour on the most common entry level planes at about $4 a gallon can add up, so say $16-36 bucks an hour in fuel. If you squirrel away about $7 an hour the whole life of an engine, you'll have enough to rebuild it when it's 2000 hours old. Of course, if you're an industry standard 100 hours a year pilot, that's twenty years of engine, assuming it's stored and maintained properly, of course. There are other costs like hangars and insurance, I pay $80 a month for a covered hangar and $800 for super duper mega insurance for the whole year. I'll know the hourly cost once I've flown for a year, the more I fly, the cheaper it is per hour.

Maintenance isn't cheap, but there's a bunch of stuff you can do yourself, preventative stuff mostly.
It really depends. Most of the guys I fly with are middle class like me. We don't make a lot of money, and we fly because we love to fly. If I had more money, I'd fly more often.
The US, I should note, is one of the cheapest places in the world to fly, second only to maybe South America or parts of the middle east (cheap fuel). Europe has expensive fuel, user fees, and extraordinary taxes that are strangling aviation, and that might be in our future too, but I hope not.