The larger thumb drives could hold a stripped down XP. I install XP Home on a 15gb partition on this computer; it takes up around 8.
Just remember that thumb drives have a different read/write system than a hard drive, which is pretty inefficient speed wise. However, many people make their own "clean-boot" Linux drives that they plug into public computers and boot from. They have basic software including office programs, Firefox, ect. This not only lets you save your work and work in familiar programs with familiar space, but it keeps the owners of the machine from watching your every move (like they do at my school). If the boot order is incorrect on the machine (or it's older than the idea of bootable flash drives) you might not be able to pull that off, though.