A general question...
When there are many players on, complicated terrain drops my frame rate considerably. Is it A) My graphic card or B) My connection. I usually run 60 FPS, when multiple cons fly through my FOV, the drop to the 30s.
I'd be pretty comfortable saying it isn't your connection. I played for years on a 26K dial-up connection, and had zero issues with many of the things people blame on connection, once I had my
computer settings worked out.
When I jumped to a much faster/better connection, my pings dropped from 225 or so down to 70 or so. I saw zero influence on the game (and on frame rate) from that.
On the other hand, some tweaks to my system made drastic, very noticeable changes to the game. I don't know if I'd blame your card though, I actually saw better results from a better Processor, RAM, and power supply. But that was because my graphics card was better than those items, but was being held back by them.
Short of replacing hardware, I think I'd look at making sure your rig is running as well as it can, with the bare minimum of background processes, etc...