Yes. As stated before, your processor is being taken away from the game.
Most likely, when you're in the tower, your processor doesn't have to work as hard to process the game data and therefore working on the background processes isn't as difficult and doesn't really take it away from the game. However, when you're flying, the processor has to perform multiple functions to track others, to track you, your flying ammunition, your movement, and the physics involved in all of this. Obviously all this work is more difficult to process and therefore makes the occasional move to take care of background processes more noticeable.