@ Caldera: Your point about retrieving accidentally deleted files from the Recycle Bin is valid to a point.
There's two caveats, though, that I can think of - with fear. The first one is to use Shift+delete, a feature I use a lot. That will bypass the Recycle Bin, as has already been said. The other scenario is deleting a file that's too big for the Recycle Bin, in which case there should appear a warning about the size and a question about directly deleting the huge file. Think about missing that warning and accidentally deleting your entire Windows folder... May this be a warning to those who may not realize the risks of disabling Windows features just because that's doable.