Couple of things might cause it. One of them is an extraneous light source in the room like sunlight. Even reflected sunlight might cause it. Without knowing what the tracking system is on your headset, if it's outside in are the towers seeing the headset or is the headset seeing the towers? Can you test that? If inside out like the RiftS the cameras don't work well in a very dark room. Make sure they have some light to work with. I have a little security cam IR light shining at the ceiling just in case.
Of course the other thing is it might just be you moving around in your seat.
Sent from my Moto Z (2) using Tapatalk