Actually, you can do two things to IE to avoid the cleanup issues. Set the cache size to something reasonable, like 20MB. Then turn on the option to empty temp files when you exit.
I have no issue with Firefox, but I do have a huge issue with Sun's Java stuff. Once you install it, you cannot remove it, without re-iinstalling the OS. I had so many problems with their Java stuff effecting so many other applications that use Java, I had to finally bite the bullet and re-install Windows.
I will never go through that again.