As I am writing this, a search for [task killer] in Android Market finds 88 apps. All these task killers come in different sizes and colours and varieties; yet their primary function is one thing: kill the apps that are running in the background. One of the first documents one would read when they learn Android development is Application Fundamentals page of Android Developer Guide. It says:
[An activity] is stopped if it is completely obscured by another activity. It still retains all state and member information. However, it is no longer visible to the user so its window is hidden and it will often be killed by the system when memory is needed elsewhere.This means that Android keeps an app in background because the user might switch back to that app in some time. This work flow would be familiar to you. You are reading an email in the Gmail app and you receive an SMS. You open the SMS from the notification bar and reply to it. When you are done with the SMS you press Back key of your phone and you are back in Gmail, in exactly the place where you left. You can continue to do what you were doing in Gmail. To make this possible, Android keeps Gmail app's state in memory.
Let's say you don't come back to Gmail, but you open a game and start playing. And let's assume this game needs a lot of memory. Now Android would kill the Gmail app automatically and reclaim the memory used by Gmail. It's given to the game you just started.
Android keeps recent apps in memory that would otherwise be left empty and kills apps and reclaims memory exactly when it is required. This lets you get back to the state in which you left an app so you can continue whatever you were doing.
All these task killers do is kill the background apps immediately discounting the possibility that you might want to go back to the same app. If you are thinking task killers make your phone faster, think again. They are actually making you slow.