It turns out that in my case removing the Nvidia Stereoscopic 3D and Nvidia 3D Vision drivers from Programs and Features completely eliminated the small pauses/freezes. If I reinstall the driver without unchecking those items the pauses reappear, and then they are gone after removing the driver for them.
I just got lazy after the beta period and let them install and then forgot about it. When I updated to the latest and greatest driver the GeForce Experience spyware came in and that got me looking.