X-Windows is the GUI for Linux. The interesting part about Linux is that it does not need a GUI to work, pretty much everything works from the command-line and it works well. So usually when I install server systems I don't even install the GUI – it saves quite a bit of memory and I can have multiple session open anyhow by pressing ALT+Fx…
If you install X-Windows you can choose from many different desktop environments. RedHat, for example, comes with GNOME by default, but you can also choose to setup KDE or others. Here is where the distributions come in, because some distributions provide more or less windows managers. Mandrake seems to support quite a few, I personally recommend "fluxbox" as a lightweight window manager.
Please note that specific desktop environments sometimes provide libraries (like DLLs) to programmers, so they don't have to program everything from scratch. This means that applications designed for one desktop environment (eg. GNOME) might not work with KDE and vice versa, unless you install the correct libraries (but this really depends on the applications and what it uses).
So is X-Windows all sunshine? It definitely has it shortcomings, especially if you compare it with the Windows GUI. It uses a lot of RAM (you should definitely consider ~128Mb of RAM if you use X-Windows, assuming you don't have any other memory consuming services running), troubleshooting and configuration (graphic drivers usually take longer since most vendors don't bother developing drivers for Linux) can be rather confusing, and the pool of applications out there is still a little bit frustrating. But – you can simply turn it off and forget X-Windows – try that with your "Windows" machine. By the way, if you want to disable the automatic startup of X-Windows, simply edit the file /etc/inittab and change the default run level from 5 to 3 there.