The performance of your system overall in AH depends most heavily on two things: The CPU and the video card. Using a lower speed CPU with a higher end video card most likely will not garner you any more video performance (and vice versa). Case in point: My Athlon XP 2100+ system. I get nearly identical frame rates and performance (at the same graphics settings) whether I use my GeForce3 TI-4200 or RADEON 9800 Pro video card, even though the RADEON is a lot more "powerful". In this case, the Athlon XP 2100+ processor is the bottle neck.
On my new Athlon 64 3200 system, the GeForce3 TI-4200 performance is nearly double what it was in the XP 2100+ system. In average graphics situations (i.e. no more than a few planes nearby) I'm getting 150+ fps with 4x anti-aliasing turned on (anisotropic filtering and mip mapping off). With the RADEON 9800 Pro, the same settings yield a fairly consistent 250-300 fps (and I've briefly seen 425+ in clear sky with nothing around). Heavy graphics drag everything down, but the RADEON will usually keep the frame rate above 100 no matter what's going on.
The RADEON on the XP 2100+ platform rarely exceeded 150 fps and was down in the 40's in heavy graphics.
So, the CPU is very important to frame rate, as much so as the video card.
System memory is important to a point. If you are running less than 512MB, then adding more memory may help, but after 512MB, you won't see much difference.
As Skuzzy pointed out, the 256MB cards use slower DDR memory than the 128MB cards...slower being a relative term here. The DDR memory used in video cards is faster than the DDR memory you use in your system. The newer video cards are using stuff on the order of a few nanoseconds.
If you are trying to decide between a 128MB video card and a 256MB card, the 128MB is the way to go. There aren't any games (that I'm aware of) that make full use of 256MB of video memory...not to say there aren't any out there, but AH is definitely not one of them. Some of the hardcore Quake gamers, or Half-life, etc may see a performance gain with the 256MB cards, but for the most part, 128MB of video memory is plenty (and faster!).
The settings you choose for your video will have a great impact on frame rate as well. When you start using anti-aliasing and anisotropic filtering, and mip mapping, things go downhill quickly. Try turning off anisotropic filtering and mip mapping and at most, use 2x anti-aliasing. If that doesn't help, turn off anti-aliasing completely as well. Set your video to lean toward performance instead of quality and you'll begin to see the trade offs. Experiment a little and you'll find a happy medium.
If you have an XP 2400+ cpu, then consider the GeForce3 TI-4200 (about $65 these days) or better card and you should have pretty good performance all around.