If you are using USB stick then I may have similar problem. From time to time, my stick will get lazy on me and the inputs will be delayed for like 5 seconds. I unplug the stick, wait a few seconds and plug it back in and it re-initializes and works just fine afterwards.
By the way, I am using the Saitek X-36.