Games are one of the most resource intensive on computer hardware. They want to run sound, graphics, networking, and device inputs all in real-time. It is a brutal environment for a computer.
Whatever you see on the screen has to get loaded into the video card memory first. The speed of the bus to and from the video card will impact the rest of the computers ability to appear to do multiple things, at the same time, and do it smoothly.
How much? That is the question. It will depend on the game and how the game apportions all its resources. Whether or not the game is CPU bound, GPU bound, or well balanced will not matter so much. Certainly a CPU bound game might show a larger advantage over a GPU bound game.