All of the arenas that aren't MA are hosted on a separate physical server. However, they're still on the same internet connection so your packet loss would still cause the disconnection.
In your case, yes, you should disconnect when doing a high-connection-stress action in any other arena.
The person sitting next to you might have a different route to the AH servers than you, defined by your ISP (especially if he is on another ISP's service). Do a pingplot on both machines to the MA server and compare them.