Did you try booting into Safe Mode? If you don't know how, press F8 repeatedly right after booting until you see the boot options menu. Navigate with arrow keys.
Unfortunately there isn't a built-in diagnostic tool set in most computers, but you can burn yourself the
Ultimate Boot CD which will contain more than you'd ever need in that category. From the CD I'd recommend Memtest 86+ for testing the RAM and the HDD diagnosis from your hard disk manufacturer (SeaTools will work for all brands).
Failing GPU and PSU can't be tested in the same manner, the only easy way is to replace them with a known good spare. Also the CPU is easiest to test that way.
A failing PSU can make all tests for other components fail.