increase dead band in game controller mapping advanced options.
If your joystick is a little loose, it could be providing very small amounts of "up" input to the game. Try increasing the dead band of that axis.
Also in advanced options for your controller, the game will give you a real time feedback of your axis, so you'll be able to see at a glance if the joystick is sending "up" or "down" inputs to the game.
I think every joystick needs a little love from the advanced controller options menus in game, just to configure it to work well for in game.
GOOD LUCK!