Memtest86 is good. When my PC has behaved like that in the past, its always been down to too much heat so it might be worth checking out fans, heatsinks.
Tweaking ram needs to be done via the bios, uping voltages, latency timings etc. AFAIK the only way to do that is by actually going into the bios and doing it by hand.