There isn't a lot of data being bounced back and forth to the servers, so capacity isn't an issue. A stable latency of <500ms puts you in the "sweet spot," but that's not always possible for everyone in the world or in the middle of nowhere.
Your ISP only gets you so far. They contract with other providers to get you to the AT&T network where the AH servers reside. All it takes is one bad router or misconfiguration along the route to affect the stability of your connection. And, some providers along that path may force asynchronous routing on the data - it goes to the servers along one route, and returns to you along another route.
Unfortunately, you have little control over that. Your only option is to use a different provider, if one is available. Many problems are wrongly attributed to a connection problem or AH servers. Many times, the problem lies with the user's machine or router/modem:
Spyware, malware, adware, improperly configured firewalls, running two firewalls (a hardware firewall in their router plus Windows firewall activated), running anti-virus software during game play (you can't get a virus from playing AH), wrong video or sound drivers or misconfigured settings, too many services and applications running in the background, etc. can be causing more problems than the connection.
