Start with your settings folder, I use a flash drive for starters.
I have a AH backup file with settings, sights, maps, terrains and other assorted goodies.
That is saved in several locations, on a DVD disk, on my wifes computer, and on the 2nd partition of this computer.
Actually if I went and looked I'd probably find 3 or 4 backups going back to 2001.
Your only limited by your imagination.