I would try auto-takeoff first, until you get the feel back.
Rudders are not very effective when there is little air flow over them, so at start off roll, they are almost useless. Roll on the throttle until you get some air speed over the control surfaces and that will help.
Or use auto-takeoff and just throttle up.
Also go to the Options->Controls->Map Controllers panel and see if any of the axises are floating (i.e. input with no stick motion). Just watch the numbers in parentheses and if any change when you are not moving the stick, then that is errant input from the stick. A little deadband added to that axis will normally smooth things out.