Quality of your Avi depends on a lot of things. Screen res, game quality settings, frame rate you record, and ending avi screen size. Fraps, in the simpliest version, takes screen shots of your screen and stores them as bmps. Fraps codec is uncompressed, its an avi in raw form. Fraps can only record to the screen size on a CRT screen, 1152 x 864. This is why your recordings can get really big. Record at 640 x 480 or if your rig can handle it 800 x 600 @ 12 fps. Make sure you have your anti virus and any programs you don't need running in the background are shut down. After you are done, take the AVI and dump it into virtual dub and increase the frame rate to 24 fps. At this time you should convert it to for the first compression, myself use HuffyUV, it's a lossless codec. Then dump it into your video editing software. After editing it, dump it back into virtual dub and compress it into whatever codec you want for the final version... I, myself use Xvid, seems to be the best at this momment.
Oh forgot... defrag your computer too before you do this.
What is the maximum resolution supported by Half/Full-size video capture?
The maximum resolution for Half-size capture is 2048x1536. Full-size video capture is limited to a maximum of 1152x864 (4:3) or 1280x720 (widescreen). If Full-size is selected and the game is set to a resolution higher than the maximum, Fraps will automatically switch to Half-size when recording.
Hope this helps.