Spamassassin Daemon
===================
The purpose of this program is to provide a daemonized version of the
spamassassin executable. The goal is improving throughput performance for
automated mail checking. This document is a brief synopsis of how spamc/spamd
work, and how to use them effectively.
Spamd
-----
Spamd is the workhorse of the spamc/spamd pair -- it loads an instance of the
spamassassin filters, and then listens as a daemon for incoming requests to
process messages. By default, spamd listens on port 783, but this is
specifiable on the command line. When spamd receives a connection, it spawns a
child to handle the request. The child will expect to read an email message
from the network socket, which should then be closed for writing on the other
end (so spamd receives an EOF). spamd will then use SA to rewrite the message,
and dump the processed message back to the socket before closing the
connection. The child process then dies. In theory, this child-forking should
be quite efficient, since on most OSes the fork will not actually copy any
memory until the child attempts to write to a memory page, and then only the
dirty page(s) will be copied. This means the entire perl engine and the SA
regular expressions, etc. will only be loaded once and then be reused by all
the children, saving a lot of overhead.
Spamc
-----
Spamc is the client half of the pair. It should be used in place of
'spamassassin' in scripts to process mail. It will read the mail from
stdin, and spool it to its connection to spamd, then read the result back and
print it to stdout. Spamc has extremely low overhead in loading, so it should
be much faster to load than the whole spamassassin program (and a perl VM).
Installation
------------
Simply copy the two executables to where you want them. Then, configure your
system to run spamd in the background, and where your mailer invokes
'spamassassin' instead invoke 'spamc'. It's that easy!
There's a Red Hat/Mandrake-style startup script called 'spamassassin'
in this directory, suitable for installation in /etc/rc.d/init.d .
Security
--------
Since spamd effectively has both read and write access on all of the mail
which passes through it , you may want to keep security in mind. Depending
on the nature of your set-up. If you are installing it on a site-wide
basis at least some caution is advisable.
System-level security:
Spamd has the facility to run as a non-root user, this has potential security
payoffs. If a fault is found in spamd or spamassassin code, any third party
linked-libraries or imported perl modules there is the potential for abuse of
both the running uid of spamd, and the uid of the username supplied by spamc
(and this could be any user).
If you do not need to let your users define their own rules, maintain their own
whitelists, or have non-world-readable home and ~/.spamassassin directories,
then just set spamd up to run with the "-u username" option. Since spamd can
use auto-whitelisting, which requires it maintain a database of email addresses
on-disk, you should use a non-"root" but non-"nobody" user: "mailnull" or
"mail" are good choices, or even create a "spamd" user.
When run as root, spamd will change uid's to the user invoking spamc in
order to read and write to their configurations. This functionality
is not possible if spamd does not run as root and is a disadvantage if
you rely on this. If you use mysql for per-user configuration there
is no reason in the world to run as root, and this remains fully
functional.
If you plan to use Razor or Pyzor, please note that they both rely on their
external configuration files in ~/.razor and ~/.pyzor being readable, and Razor
will try to write to a log file in ~/.razor/razor-agent.log that must be
writable (Razor will complain about 'unblessed references' in this case). You
may find the -H switch to spamd to be useful; it allows you to set a 'helper
home directory' that will be used as $HOME when external helpers like Razor,
Pyzor and DCC are run.
Since spamd makes no effort to authenticate the username supplied by
spamc, it is easily possible for malicious users invoking modified
spamc clients to make spamd:
(1.) read (and hence determine) the contents of other users configurations
(2.) change the contents of other users configurations (whitelisting)
(3.) grab CPU time as that user -- this is an issue on ulimit'd systems
If users do not have the opportunity to invoke spamc themselves, and
the network is secure, running spamd as root is the preferred option,
Be clear that the issues above dont affect you. Note: if you use mysql
for per-user configuration on systems, you will remain vulnerable to
(1.) and (2.).
configuration: Mysql .spamassassin/user_prefs
/ \ / \
/ \ / \
can users connect? yes no yes no
| | / |
| | / |
unsafe | / root prefered
| /
safe as non-root
|
|
some deprecation
Network security:
If you use spamd across a network and spamc connects from other hosts,
you should ensure (as with all services) the security of your network
segments. Mail is sent as plaintext, and is prone to packet sniffing and
spoofing techniques if you are on an insecure network. If you cannot avoid
this consider using an encrypted transport layer, such as a VPN, ssh
tunnel or similar.
Performance
-----------
So how much faster is this than just using 'spamassassin'? Well, on my 400MHz
K6-2 mail server, spamassassin process a 11689 byte message in about 3.36
seconds, spamc/spamd processes the same message in about 0.86 seconds, or about
4 times faster. With bigger messages, the difference is less pronounced; a
115855 byte message takes about 5 seconds with spamassassin, and 2.5 seconds
with spamc/spamd, or about 2 times faster. However, if many messages are being
processed in parallel, the spamc/spamd combination will likely be much more
efficient, since spamassassin has much higher overhead starting up, and will
consume more non-shared memory than will spamc/spamd. For example, on the
115855 byte message, spamc consumes *no* heap memory (and very little on the
stack), where spamassassin uses over 15MB of heap space and a peak of 3.5M.
In processing the 115855 byte message 10 times in parrallel, spamd uses just
22M of heap, with a peak of only 2.5M spamassassin would have used 150M
total, and a peak of up to 35M to do this same job.
Bugs
----
There are no known bugs with this setup. Several reasonable sized sites are
now running it on their production mail systems. However, you should still
test it completely in *your environment* before trusting all your mail to it.
If you discover compilation, runtime, or load-performance bugs, please notify
the SpamAssassin-Talk mailing list (SpamAssassin-Talk@lists.sourceforge.net)
Network Protocol
----------------
The protocol for communication between spamc/spamd is somewhat HTTP like. The
conversation looks like:
spamc --> PROCESS SPAMC/1.2
spamc --> Content-length: <size>
(optional) spamc --> User: <username>
spamc --> \r\n [blank line]
spamc --> --message sent here--
spamd --> SPAMD/1.1 0 EX_OK
spamd --> Content-length: <size>
spamd --> \r\n [blank line]
spamd --> --processed message sent here--
After each side is done writing, it shuts down its side of the connection.
The first line from spamc is the command for spamd to execute (PROCESS a
message is the command in protocol<=1.2) followed by the protocol version
The first line of the response from spamd is the protocol version (note this is
SPAMD here, where it was SPAMC on the other side) followed by a response code
from sysexits.h followed by a response message string which describes the error
if there was one. If the response code is not 0, then the processed message
will not be sent, and the socket will be closed after the first line is sent.
Commands
The following commands are defined as of protocol 1.2:
CHECK -- Just check if the passed message is spam or not and reply as described below
SYMBOLS -- Check if message is spam or not, and return score plus list of symbols hit(*)
REPORT -- Check if message is spam or not, and return score plus report(*)
SKIP -- Ignore this message -- client opened connection then changed its mind
PROCESS -- Process this message as described above and return modified message
(*) These methods are not supported as yet by spamc
CHECK command returns just a header (terminated by "\r\n\r\n") with the first
line as for PROCESS (ie a response code and message), and then a header called
"Spam:" with value of either "True" or "False", then a semi-colon, and then the
score for this message, " / " then the threshold. So the entire response looks
like either:
SPAMD/1.1 0 EX_OK
Spam: True ; 15 / 5
or
SPAMD/1.1 0 EX_OK
Spam: False ; 2 / 5
SYMBOLS command returns the same as CHECK, followed by a line listing all the
rule names, separated by commas.
REPORT command returns the same as CHECK, followed by a report generated by
spamd.
Shared Library
--------------
spamc can now be used as libspamc.so; simply run "make spamd/libspamc.so" at
the top level. Thanks to Liam Widdowson <liam@inodes.org> for this patch.