Heat heat heat! Try taking the side off and blowing in a desk fan. If it is stable, get more fans. I spent months troubleshooting a problem like this and couldn't believe it was heat. It was! Now I have no problems at all. I drove everyone on the AH BBS insane with my whinging, blaming drivers, DX, my AH settings file, Windows, RAM timings, my poor benighted Hotas Cougar (innocent, though not totally).
Good luck. These problems are NO FUN;)