Try reducing the rudder input sensitivity, gradual drifts could be just because your resting your feet/wrist without intending any input to be made through the devices, but their sensitivity is up so that it detects it and starts turning your vehicle.