Close Chalenge. The 32bit OS does have access to the entire 4GB address space, but you instantly lose system RAM space due to all ROMS, I/O address space, I/O buffers (i.e. any RAM associated with sound cards, ethernet cards, and so on), and video card RAM.
Video card RAM can be paged in. It is more inefficient to do so, but with cards that have a large amount of RAM (i.e. >512MB) that is what the drivers can do in order to prevent too much loss of system RAM. However, the video drivers also grab a chunk of system RAM to use while they are paging in the video card RAM. It really is quite messy.
Now, here comes the freight train. With a 64bit OS, the video card RAM will get mapped to the last address space the OS will address (say 16TB), which is a long way from the physical system RAM space. The exception is for 32 bit applications. 32 bit applications have to have the video memory space still located in first 4GB of system RAM space.
It is less of an issue as the 64bit OS can allocate itself and anything else not directly addressable by the 32bit application to address space above 4GB which allows more physical space for the 32bit application.
Once you get into video cards with more than 512MB of video RAM, it is far more efficient to run a 64bit OS, such as Windows 7. The 64bit version of Windows 7 is, currently, the most stable and best supported 64bit OS. It still has some issues with flooding any given application with "USB Device Change" messages if the USB bus power drops too low, but that can be worked around using an externally powered USB hub.