Valid XHTML 1.0 Strict
Valid CSS!

[ Docs | Tools | Advisories | Full-Disclosure ]


This document clearly outlines the steps required to create a copy of Sendmail that implements the SMTP AUTH command (see RFC 2554), using the SASL library (see RFC 2222).
This approach has been verified to work with a number of mail clients, including Microsoft Windows clients such as Outlook Express and Pegasus Mail.

Revision History:

2001-06-11 Initial Revision
2002-01-18 Added Sendmail 8.12.x/Solaris 8 information
2002-03-29 Added Server-Server AUTH information


1. Download the latest version of Sendmail and the Cyrus SASL library

... available from:

2. Take some backups before we start!

cp /etc/mail/ /etc/mail/
cp /usr/lib/sendmail /usr/lib/sendmail.old

3. Build the SASL library and install it

gzip -cd cyrus-sasl.1.5.24.tar.gz | tar xvf - 
cd cyrus-sasl.1.5.24 
./configure --enable-login --prefix=/opt/local  -includedir=/usr/include -libdir=/usr/lib --with-dbpath=/etc/mail/sasldb
make install
cd ..

(Updated: 2002-03-29) Note that if you are building on Solaris 8 and need to use the gss api or kerberos, you have to install Sun's supplemental encryption packages.
SASL seems to require the following libraries in that case: libgssapi_krb5, libkrb5, libk5crypto and libcom_err.
You can disable gss and kerberos by specifying: --disable-krb4 --disable-gssapi, if your objective is to merely provide a somewhat secure relay mail server.

The --enable-login command makes the SASL library compatible with Outlook Express, but enables a rather weak 'encryption' scheme in this case.

As Sun Microsystems recommend, we refuse to use the default installation path of /usr/local, so we put everything into /opt/local instead.
We choose to install the libraries in a standard location, however, so that Sendmail does not have trouble finding them at runtime.
However, SASL will then expect to find the libraries in /usr/lib/sasl. The easiest workaround is to create a symlink as follows:

ln -s /opt/local/lib/sasl/ /usr/lib/sasl

4. Create your site.config.m4 file

Unpack Sendmail:

gzip -cd sendmail.8.11.4.tar.gz | tar xvf -
cd sendmail.8.11.4

Choose the definition from the devtools/OS folder that most closely matches your target system, and copy it to devtools/Site/site.config.m4.

For example, on a Solaris 2.6 machine:

cp devtools/OS/SunOS.5.6 devtools/Site/site.config.m4

5. Add SASL support to Sendmail

Edit the site.config.m4 file that you just created, and edit it as follows:

vi devtools/Site/site.config.m4

Change the confENVDEF statement to include -DSASL

Change the confLIBS statement to include -lsasl

Now is also the time to add any extra features you might want to compile in, such as TCP Wrapper support.
For example, on a Solaris 2.6 machine with SASL and TCP Wrappers enabled, the file would look like this:

define(`confCC', `gcc')
define(`confBEFORE', `sysexits.h')
define(`confLIBS', `-lsocket -lnsl -lkstat -lwrap -lsasl')
define(`confMTLDOPTS', `-lpthread')
define(`confMBINDIR', `/usr/lib')
define(`confEBINDIR', `/usr/lib')
define(`confSBINGRP', `sys')
define(`confINSTALL', `${BUILDBIN}/')
define(`confDEPEND_TYPE', `CC-M')
        if [ -r /usr/include/sysexits.h ]; 
                ln -s /usr/include/sysexits.h; 

(Updated: 2002-01-21) For Sendmail 8.11.6 on Solaris 8 we used:

./Build -I/opt/local/BerkeleyDB/include -I/opt/local/include -L/opt/local/BerkeleyDB/lib -L/opt/local/lib

Our site.config.m4 looked like:

define(`confCC', `gcc')
define(`confLDOPTS_SO', `-G')
define(`confBEFORE', `sysexits.h')
define(`confENVDEF', `-DSOLARIS=20800 -DNETINET6 -DSASL')
define(`confLIBS', `-lsocket -lnsl -lldap -lsasl')
define(`confMTLDOPTS', `-lpthread')
define(`confMBINDIR', `/usr/lib')
define(`confEBINDIR', `/usr/lib')
define(`confSBINGRP', `sys')
define(`confINSTALL', `${BUILDBIN}/')
define(`confDEPEND_TYPE', `CC-M')
        if [ -r /usr/include/sysexits.h ]; 
                ln -s /usr/include/sysexits.h; 

(Updated: 2002-01-21)For Sendmail 8.12.1 on Solaris 8, our site.config.m4 looked like this:

VERSIONID(`$Id:,v 8.13 2001/06/27 21:46:30 gshapiro Exp $')
define(`confPRIVACY_FLAGS', ``authwarnings,noexpn,novrfy'')dnl
FEATURE(`local_procmail', `/opt/local/bin/procmail')
FEATURE(`access_db', `btree -T<TMPF> /etc/mail/access')dnl
FEATURE(`virtusertable', `btree -T<TMPFgt; /etc/mail/virtusers')dnl
define(`confMAX_QUEUE_RUN_SIZE', `40000')dnl
define(`confMAX_MESSAGE_SIZE', `10000000')dnl
define(`confSEPARATE_PROC', `False')dnl
define(`confSINGLE_THREAD_DELIVERY', `False')dnl
define(`confTO_IDENT', `0')dnl
define(`LOCAL_MAILER_FLAGS', `/59:@ADFMflmnqsw|')

Notes: We use procmail as local delivery agent. If you don't, then just delete the following:

FEATURE(`local_procmail', `/opt/local/bin/procmail')
define(`LOCAL_MAILER_FLAGS', `/59:@ADFMflmnqsw|')

... since the default local mailer and flags will work fine for your OS.

This config would be suitable for a pre-8.12 release of Sendmail if run as root - simply remove:


6. Now compile Sendmail and check to see if SASL is working OK

cd sendmail
sh Build
cd ../obj.SunOS.5.6.sun4/sendmail/
./sendmail -d0.1 -bt < /dev/null

You should see output similar to this:

Version 8.11.4
Compiled with: 

Note the SASL, which indicates that the library was built into Sendmail correctly.

7. Configure Sendmail

Here we create our configuration file, starting by choosing the most appropriate default file, then tweaking it until it suits us 100%. In our example, we choose the Solaris 2.x configuration as our base.

cd ../../cf/cf

Then we add a few options to suit our tastes, but the important part as far as getting the SMTP AUTH command to work are these lines:


This says the following: Allow users to authenticate using with the DIGEST-MD5, CRAM-MD5, or LOGIN protocols.
The second m4 command says, 'trust any user who has authenticated themselves using one of these protocols.'

Now we're ready to build the file from the m4 source:

sh Build

8. Test the new Sendmail configuration

The will soon be installed into /etc/mail/, but before we do that, we have one last testing opportunity:

../../obj.SunOS.5.6.sun4/sendmail/sendmail -C -bt -d0.15 < /dev/null

If you like what you see, then you can install Sendmail and the new configuration file as follows:

/etc/init.d/sendmail stop
cp /etc/mail/
cd ../../sendmail/
sh Build install
/etc/init.d/sendmail start

If everything has gone to plan, your shiny new Sendmail daemon should be running, and you're ready to test the SMTP AUTH feature.

9. Create users and test the SMTP AUTH command

The next step is to populate /etc/mail/sasldb with usernames and passwords. These are added using the saslpasswd utility like this:

/opt/local/sbin/saslpasswd -u yourrealm test
Password: ****
Again (for verification): ****

We can list the users in this file with the sasldblistusers program:

user: test realm: mech: DIGEST-MD5
user: test realm: mech: CRAM-MD5
user: test realm: mech: PLAIN

This shows us that the 'test' user can use SASL with any one of three authentication methods. We now need to check that Sendmail is ready and waiting to accept the SMTP AUTH command:

telnet localhost 25
Connected to localhost.
Escape character is '^]'.
220 ESMTP server ready
EHLO foo
250- Hello localhost [], pleased to meet you
250-SIZE 10000000
250 HELP
221 2.0.0 closing connection
Connection closed by foreign host.

As you can see, the 250-AUTH command was returned, with a list of the valid authentication methods for this server. We're ready to set up an email client now.

10. Mail Client Configuration

This varies greatly between clients, but the principles are always the same. Find the configuration for your SMTP server, and select 'this server requires authentication' or the similar option.

The username will be formed from the name and 'realm' shown in the output of sasldblistusers: for example, in our previous output listing the username would be
There is an option to saslpasswd, -u DOM, that lets you change the realm (or domain part of the above address).
This is useful if you want to set the SMTP username and password to be the same as the POP password for certain users.

The password will be whatever you entered into saslpasswd before.

Once this is set up, try to send a message to yourself, and once it comes back to you, check the headers:

Received: from johnc ( [])
by (8.11.4/8.11.4) with ESMTP id f56dvg972875
for <>

The presence of the 'authenticated' header means that everything has been installed and is working successfully.

11. Server to Server authentication

It is also possible to use SMTP AUTH to authenticate between servers. This is perhaps most useful when used to allow a mail relay to authenticate to a central mail hub machine. The configuration is fairly straightforward:

Firstly, on the server that will be authenticated to - i.e. the true 'server' in the client->server authentication - you must create an account with saslpasswd as you would a regular user account:

/opt/local/sbin/saslpasswd -u foo mailrelay
Password: ****
Again (for verification): ****

This is all that need be done on this side of the connection.
Next, we must configure the authenticating server (or 'client').

the file must be altered to include the following entry:

FEATURE(`authinfo',`btree -T<TMPF> /etc/mail/authinfo')  

or similar - in our example we choose to use btree, but dbm, etc are equally suitable. This instructs Sendmail to use an external AuthInfo map, located in /etc/mail/authinfo.

Following this change you must regenerate your in the usual manner. The next step is to populate this file so that we may generate a map.
In our example, will authenticate to with realm 'foo' and username 'mailrelay'. The corresponding AuthInfo file is shown below: "U:mailrelay" "I:mailrelay" "P:secret" "R:foo" "M:DIGEST-MD5"

The line reads as follows: for server '', use (U)sername 'mailrelay', authentication (I)d 'mailrelay', (P)assword:'secret', (R)ealm 'foo', and (M)ethod 'DIGEST-MD5'. You need only specify a 'U' or 'I' entry - they are interpreted the same way by the server.

Following successful authentication, you will notice a log message such as:

Mar 29 16:07:50 mailrelay sendmail[3097]: [ID 702911] AUTH=client,, mech=DIGEST-MD5, bits=128

That is basically all that is required to get server-server authentication set up. This allows you to remove your relay machines from your access maps and force them to authenticate in the same way your users do.

Another great use for this is to provide an audit trail through your mail system. I like to have entries on every server so that they can all authenticate with each other (by adding saslpasswd accounts and AuthInfo maps to every appropriate Sendmail installation.)

12. Troubleshooting your installation

Your best friend when troubleshooting is the log file. Start up Sendmail with a high LogLevel, e.g.:

/usr/lib/sendmail -bd -OLogLevel=14

... And watch for errors in the appropriate log file (typically /var/log/syslog in the case of Solaris, or /var/log/maillog elsewhere)

Common errors include unsafe files (chmod g-w / /etc /etc/mail /usr /usr/lib) and similar file permission issues that are usually quick to resolve. If you see something like:

Jan 21 02:02:07 yourhost sendmail[10558]: [ID 801593 mail.warning] g0L727a10558: AUTH failure (LOGIN): user not found (-20)

Then you probably forgot to use a realm when creating the sasl user account..

One problem that has reared its ugly head on several occasions is the inclusion in the logs of messages like this:

Unable to dlopen /usr/lib/sasl/ /usr/lib/sendmail: fatal: relocation error: file /usr/lib/sasl/ symbol des_ecb_encrypt: referenced symbol not found

The solution here is to rebuild the SASL library without support for DIGEST-MD5. This can be accomplished by repeating step 3 with the addition of the option --disable-digest. Kerberos (use --disable-krb4) can cause similar problems too.

A tell-tale sign that you have library problems is that when running sasldblistusers, not all the authentication mechanisms you defined are listed. For instance, if DIGEST-MD5 didn't work correctly, the output would just look like:

user: test realm: mech: CRAM-MD5
user: test realm: mech: PLAIN

Rebuilding SASL is definitely the quickest fix for these problems. However, one caveat is that you must then remove the appropriate auth mechanisms from the Sendmail configuration, so repeat step 7, altering as appropriate, before copying it over /etc/mail/

As far as client support goes, you should be able to support 99% of mail clients using CRAM-MD5, while the remainder use PLAIN, so the DIGEST-MD5 mechanism isn't going to be mission-critical in most cases.


SMTP AUTH is a useful command that allows you to maintain a closed mail relay whilst allowing selected mails to bypass this protection.
Although there are a few potential problems, hopefully this article will assist you in implementing your own authenticated SMTP server, and fixing it if it goes wrong.

John Cartwright <>