When I used Windows, I had come across several “system cleaners and optimizers”. One I liked was CCleaner (Crap Cleaner). It had a nice interface and was capable of cleaning cache and data of several applications.
I found a similar application for my Ubuntu 10.04 Lucid Lynx Alpha. It has similar interface and features. It is called bleachbit and can be installed via Synaptic (universe repos need to be enabled) and can be accessed from Applications>System Tools>Bleachbit. It is also capable of wiping the data so that it is impossible(or lets say difficult) to recover. It can also clean backup files (those hidden with ~ at the end of the filename, .bak, .tmp and more) with Deep Scan.