It depends on the game configuration/settings. The game can be configured to run on a video card with 256MB of video RAM. It can also be configured to need 3GB of video RAM.
Frame rate drop can be from many things. With the current game it can be due to the CPU and video card GPU. Typically it is the GPU on the video card and not the video RAM that causes the frame rate drops.
However, it can be also due to resource starvation in any of the computer systems (sound, networking, video, CPU, system RAM....).
A DXDIAG output would go a long way in helping to determine what game settings would be best for your system.