I need to do some testing because I'm still confused. So if I trim with my keys, or in my case my stick buttons, and hit combat trim on it goes back to the original settings until I trim again?
If you turn CT on, it'll trim you out. When you shut it off, your trims will stay were CT left them (even if that isn't the best place for them to be). If you manual trim with buttons or keys, you'll "lose" your trim settings when you turn CT on (or auto-level, auto-angle, or auto-speed/climb).
When you turn CT off again, you'll need to readjust your trims. You can do that by memorizing where you prefer to have the trim indicators, or you can fly at your desired speed and adjust them for straight/level. As your speed changes, your trim will change.
As TC mentioned, I wouldn't worry about that while in a fight.
If, however, you have your trims mapped to rotary dials or sliders, you can "pre-set" your trims, and they'll return to those settings anytime CT is disabled. On my X52, I actually set the trims for straight/level at my desired speed, and then take a Sharpie marker and mark the side of my dials so if I bump them I can return them to my desired setting while still using CT (or, I can visually verify that they're set right at any given time).
IMO, the rotary dial option is best, but the downside is that you need to calibrate your dials anytime you calibrate your sick in AH. If you forget, they won't work right until you re-calibrate them.
Having trim mapped to rotary dials is nice, because it allows me to put my plane into a "familiar" trim-state, where I know how everything "feels" and responds. The key for me was to have CT off by default, and turn it on when I want it. Having it on by default and shutting it off when I didn't want it didn't work nearly as well. If I bump the toggle or get confused for a moment, I don't want CT taking over...
CT is a great tool for the game, and makes flying much more enjoyable. However, it really only works properly for "basic" flight, clean, at medium speeds, and full throttle. When speed gets high or low, or gear or flaps come out, or you're "roping", etc, manual trim works better and gives you an edge (once you get proficient at it) IMO.
CT won't throw you into the ground, but it will "fight" you at times. For example, try landing with CT, and then landing without CT. With it on, you'll need aileron and elevator input to overpower the CT trim settings. You'll be approaching the runway with the stick in the corner...
With CT off, and trimmed manually, you can approach with your hand off the stick... The plane almost lands itself! There's a big difference when you're trying to RTB while missing big parts of your plane too... You'll trim to allow you to fly home, and then need to re-trim when you drop flaps and gear.