A friend of mine used to (and probably still does) make of Linux – "Need to add a new mouse to Linux? Just recompile the kernel and you're all set!". Recompiling the kernel might have been difficult and tedious at some point, but it's really only typing in a few commands and rebooting.
The kernel is the core of Linux, the workhorse. Memory management, process scheduling (yes, multiple processes need to be scheduled!), file access and such are all things the kernel does.
Now why do we want to recompile the kernel? Most users actually might not want to recompile the kernel; due to the introduction of modules most installations ship a general core kernel (that supports most devices needed to boot) and provide the rest through modules. This means that you save memory since only the necessary modules are loaded and also means that you most likely will need to change or customize the kernel.
Compiling your own kernel can come in pretty handy though sometimes, old machines that have little memory will definitely benefit, exclude support for security sensitive features, use the latest stable version and so on.
So how do you compile the kernel? Navigate to http://www.tldp.org and look for the "kernel-howto" there.