The big problem I see with changing times is that you're messing with something that isn't broken.
There are 500+/- folks playing in this timeframe. A few years ago it was 150-200.
If you alter the time, you're going to get backlash from all those established folks who've been playing for years suddenly having to alter their schedules or miss out due to their own conflicts.
So then you're stuck with having to make a choice between those that want to but cant due to their schedules, and those that have been for years but now can't because of their schedules. Sounds like a textbook 'no win situation' to me.
Who's to say that the numbers won't increase if you do change the times? No way to tell until you try it. But, in the meantime, your little experiment could cause a lot of permanent damage to the existing cadre.
It is unfortunate that a time that satisfies everybody cannot be chosen. I have to burn vacation time just to watch my daughter play volleyball because the schools insist on scheduling the games right after school.. which is about 2 hours before I get off work.