Mail transport agents are the software that transfers mail from your local system to remote systems. It is very seldom necessary to mess with or replace your MTA on a modern Linux, and you're better off not fixing what isn't broken. Nevertheless, here's a survey to get you started on understanding what the tradeoffs are if you decide you need more security or performance than your system's default can offer.
(There are other Unix MTAs besides these, but you are quite unlikely to encounter them on a Linux box.)
Each has its own unique features, but the best compromise is qmail. It features high security (even if vmail is more secure), high speed (even if smail is faster for local uses) and ease of configuration. Of course, feel free to choose any mail software. The information provided here is intended to help you choose well.
Sendmail can be nice for many sites with complicated options, but I think its configuration is too hard for beginners while it is not very secure or very fast, so there is only a really outdated sendmail section in this HOWTO.
If you know what you're doing, choose sendmail (and you shouldn't be reading this HOWTO!); otherwise I generally recommend qmail.
Detailed descriptions of these programs follow.
BSD sendmail is the grandaddy of Internet MTAs. It has outlasted a few would-be successors. Most Linux distributions now use it and have it preinstalled.
Sendmail has a long-standing reputation for being an administrator's nightmare -- hard to understand, tricky to configure, rife with security holes. As Internet technology and standards have stabilized, however, many of the sendmail options and configurable rules that gave rise to this reputation have ceased to require per-site tweaking (the effective demise of non-TCP/IP network layers like UUCP has helped a lot). Also, recent sendmail versions have an improved configuration system that insulates you from the legendary hideousness of the sendmail.cf configuration file. Most importantly, sendmail now normally comes preconfigured, and you should never need to touch it unless you have unusual requirements (such as needing to route mail over a non-TCP/IP network).
There is a sendmail home page at http://www.sendmail.org. It includes references to extensive documentation of sendmail, should you actually need to wrestle with custom-configuring it.
Other MTAs, if called as `sendmail', may mimic the semantics of sendmail's command-line options. This is convenient for mail user agents, which often assume they are talking to sendmail.
Smail was the first serious attempt to replace sendmail. It has a simpler and much more comprehensible configuration system than sendmail's, and it's fairly secure. Some Linux distributions preinstall it rather than sendmail.
At one time smail's excellent support for mixed TCP/IP and UUCP sites was a major selling point for it, but as UUCP has declined, so has smail. Also, smail is less efficient than sendmail on high-volume connections.
As with sendmail, it is unlikely that you will need to tweak a preinstalled smail configuration.
(Very occasionally you might run across references to an `smail 2.5'. This program has been obsolete for a long time. Don't bother with it.)
The qmail program is a sendmail-compatible MTA designed specifically for high security. The author has a standing reward of $500 for publication of the first verifiable security hole; this reward has gone unclaimed since March 1997.
The qmail home page is at http://pobox.com/~djb/qmail.html.
The exim program is similar to smail3, but with more features. It advertises particular strengths in spam-blocking and support of several virtual hosts (virtual DNS domains) on the same host.
The exim home page is at http://www.exim.org/.
I tried it on my own computer, it looks like a nice merge between smail configuration system and qmail security, moreover it has the advantage of being GPL.
A section explaining how to replace your current MTA by exim will be added soon.