Two aspects of flying without combat trim is diving survival and vertical maneuvers. CT works based on the speed of the aircraft so in a steep dive, say, when you're dive bombing for example, with CT on, you continually get nose down pitch trim until it hits the stops. This makes your elevator less responsive on the pull-out and usually forces me to make a hard manual pitch trim up adjust as I'm beginning to pull up. The second is during vertical maneuvers. As you zoom climb, your speed slows, and CT starts adding nose-up pitch trim. As you stall out and come over the top, you'll have an extreme amount of nose-up trim in the aircraft, and will fight the nose-up tendency as you accelerate coming back down. This is one reason you'll find most of the hardcore P-38 guys never use combat trim--very little lateral trim necessary due to the counter-rotating props that minimize lateral instability. Then, they only have to manipulate the pitch trim to maximize their performance, either turning or in the vertical.
There are aircraft that I can't seem to keep trimmed up very well--mostly the high torque variety such as the 109's, Spit14, etc. When I fly these aircraft, I usually manually engage combat trim. After I started flying the P-47 quite a bit, I noticed it was very stable laterally, and required virtually no rudder or aileron trim once you were level and in cruise. So, I started flying with no combat trim in most of the American rides as a result. I almost never auger now coming out of dives, and when I do, its mostly when I have CT engaged. Furthermore, you can get the most out of your zoom climbs and vertical stall maneuvers if CT doesn't give you full up pitch trim due to your speed low.
From a hardware standpoint, I have a CH Throttle which has pitch and rudder trim mapped to a hatswitch. I find I don't use aileron trim that much, unless I'm missing wingtips. When I finally get my CH Fighterstick, I'll have all three axes mapped to get my hands off the keyboard.