Deja,
As I understand it that little square slug is not the CPU itself, but rather a slug of ... well something ... that sits directly on top of the die to help transfer heat. A little wet sanding will not kill it, but you have to use a very fine grain and be very careful.
BTW, a good way to do the sanding is with a piece of glass as a backing for the fine paper. You can normally get glass that is as close to "perfectly flat" as you can find easily. The main reason to do anything at all to the heat slug on the CPU is to remove some of the lettering and such that you do find on these CPU heat slugs. The only thing that typically needs to be smoothed and flattened to any degree is the heatsink part of the combination.
All of that aside, this isn't something I would recommend unless you are experienced, very careful, have the right tools, and are willing to buy a new CPU if you botch it.

For most folks, a $7 tube of Arctic Silver applied properly is a far better solution.
Wardog,
Have you had much experience mating the HSF and CPU with no grease without sanding anything? That would make me nervous, but I've never tried it that way. If you have I'd love to hear your results.

------------------
Sean "Lephturn" Conrad - Aces High Chief Trainer
A proud member of the mighty Flying Pigs
http://www.flyingpigs.comCheck out
Lephturn's Aerodrome for AH articles and training info!