Of course, you can drag a file to the Recycle Bin and empty the bin, but all this really does is clear the icon, remove the file's name from on your computer, and tell Windows that it can use the space for something else. Until it actually does use that space, the space will be occupied by the contents of the deleted information, much like drawers of files with labels on each drawer, what happens is removing the label but the drawer still contain the files. This is why, if you have the right software and act quickly enough, you can restore information that you've deleted by accident.
You should also keep in mind that files are created and insecurely deleted, without your knowledge, every time you use your computer. Suppose, for example, that you are writing a large report. It may take you a week working several hours each day, and every time the document is saved, Windows will create a new copy of the document and store it on your hard drive. After a few days of editing, you may have unknowingly saved several versions of the document, all at different stages of completion. Windows generally deletes the old versions of a file, of course, but it does not look for the exact location of the original in order to overwrite it securely when a new copy is made. Instead, it simply puts the latest version into a new place of the metaphorical drawer mentioned above, and moves the label from the old drawer to the new one, and leaves the previous draft where it was until some other program needs to use that space.
So obviously, if you have a good reason to destroy all traces of the documents from your drawers, removing the latest copy is not going to be enough.
What to do? Wiping information with secure deletion tools
You need to you use a secure deletion tool, it would be more accurate to say that you are replacing, or 'overwriting' your sensitive information, rather than simply deleting it. If you imagine that the documents stored in those metaphorical drawer discussed above are written in pencil, then secure deletion software not only erases the content, but will scribble over the top of every word. And, much like pencil lead, digital information can still be read, albeit poorly, even after it has been erased and something has been written over the top of it.
Because of this, the tools recommended here overwrite files with random data several times. This process is called wiping, and the more times information is overwritten, the more difficult it becomes for someone to recover the original content.
Software Recommended: Eraser and CCleaner. Please go through the Help manual before installing and using them.
*Reference: documents prepared jointly between Tactical Technology Collective and Front Line Foundation