The relationship is not between your connection quality and your framerate, it's between the number of objects in the world and your framerate. When you are online, there are a lot more things in the world that the CPU has to process, so the framerates will be lower.
Did you turn the ground clutter off? That helps me a lot. Other than that, just turn the framerate counter off in the game and don't worry about it unless you are noticing real display slowdowns during a fight. It sounds like your framerate is just fine most of the time.