TipJar::MTA - outgoing SMTP with exponential random backoff.
use TipJar::MTA '/var/spool/MTA'; # must be a writable -d # defaults to ./MTAdir $TipJar::MTA::interval='100'; # the default is 17 $TipJar::MTA::TimeStampFrequency='35'; # the default is 200 $TipJar::MTA::AgeBeforeDeferralReport=7000; # default is 4 hours $TipJar::MTA::MyDomain='peanut.af.mil'; # defaults to `hostname` # bouces to certain matching addresses can be suppressed. @TipJar::MTA::NoBounceRegexList = map { qr/$_/} ( '^MDA-bounce-recipient\@tipjar.com$', '^RAPNAP\+challenge\+sent\+to\+.+==.+\@pay2send.com$' ); # And away we go, TipJar::MTA::run(); # logging to /var/spool/MTA/log/current
alternately,
use TipJar::MTA '/var/spool/MTA', 'nodns'; # we are sending to # a restricted set of domains # or using a smarthost %TipJar::MTA::SMTProutes = ( SMARTHOST => 'smtp_outbound.internal', # smarthost for forwarding, can be a list too 'example.com' => # mail to example.com will be randomly routed through these three [qw/ smtp1.example.com smtp2.example.com backup-smtp.example.org /], 'example.net' => 'bad-dog.example.net' # all mail to example.net goes to bad-dog );
On startup, we identify the base directory and make sure we can write to it, check for and create a few subdirectories, check if there is an MTA already running and stop if there is, so that TipJar::MTA can be restarted from cron.
We are not concerned with either listening on port 25 or with local delivery. This module implements outgoing SMTP with exponentially deferred random backoffs on temporary failure. Future delivery scheduling is determined by what directory a message appears in. File age, according to stat(), is used to determine repeated deferral.
stat()
We reuse a socket to a domain if we had trouble connecting to the MX for that domain in the past, or for multiple new messages going to the same domain. We also cache 4XX and 5XX error codes on recipients for four hours to eliminate a mess of traffic when, for instance, we have to bounce many messages to the same bogus return address. We will get a "550 User Unknown" error on the first bounce and throw away the others.
Every $interval seconds, we fork a child process.
$interval
A new child process first goes through all new outbound messages and expands them into individual messages and tries to send them. New messages are to be formatted with the return address on the first line, then recipient addresses on subsequent lines, then a blank line (rather, a line with no @ sign), then the body of the message. The TipJar::MTA::queue module will help compose such files if needed.
As of version 0.30, multiple recipients on the first line of a queued file will all be attempted, to the mx of the domain of the host in the first recipient.
Messages are rewritten into multiple messages when they are for multiple recipients, and then attempted in the order that the recipients appeared in the file.
After attempting new messages, a child process attempts all messages in the "immediate" directory.
After attempting all messages in the immediate directory, a child process moves deferred messages whose times have arrived into the immediate directory for processing by later children.
Deferred messages are stored in directories named according to when a message is to be reattempted. Reattempt times are assigned at requeueing time to be now plus between three and five quarters of the message age. Messages more than a week old are not reattempted. An undeliverable message that got the maximum deferrment after getting attempted just shy of the one-week deadline could conceivably be attempted for the final time fifteen and three quarters days after it was originally enqueued. Then it would be deleted.
An array of regular expressions can be specified, and if any of them match the sender of a bouncing message, the bouncing is suppressed, so you don't have to waste time with bounce messages from bad addresses you're sending challenges to for instance.
The format for new messages is as follows:
The first line of the message contains the return address. It can be bare or contained in angle-brackets. If there are angle brackets, the part of the line not in them is discarded.
All recipients are listed each on their own line. Recipients must have at-signs in them.
The first line (after the first line) that does not contain a @ symbol marks the end of the recipients. We are not concerned with local delivery.
Follow the routing information with the data, starting with header lines.
None.
the dnsmx() function uses Net::DNS. Versions 0.14 and previous use djbdns' dnsmx tool if that's preferable for you -- the old function is commented out.
dnsmx()
The file system holding the queue must support reading from a file handle after the file has been unlinked from its directory. If your computer can't do this, see the spot in the code near the phrase "UNLINK ISSUE" and follow the instructions.
For that matter, we also generate some long file names with lots of dots in them, which could conceivably not be portable.
beginning with version 0.20, the dependency on Net::DNS can be skipped by including the term "nodns" on the use line, after the MTAdir, which must appear, to avoid changing the interface. When nodns is declared, all MX lookups will be directly from the %TipJar::MTA::SMTProutes hash of array references keyed by lowercased domain names. If the desired domain does not appear, the reserved domain name 'SMARTHOST' is looked up as a fallback or a list of fallbacks. If no SMARTHOST is declared, an error will be thrown.
%TipJar::MTA::SMTProutes
The smtproute is selected from the listed routes randomly.
threw away some inefficient archtecture ideas, such as per-domain queues for connection reuse, in order to have a working system ASAP. Testing kill-zero functionality in test script.
logging to $basedir/log/current instead of stdout, unless $LogToStdout is true. $AgeBeforeDeferralReport variable to suppress deferral bounces when a message has been queued for less than an interval.
slight code and documentation cleanup
Testing, testing, testing! make test on TipJar::MTA::queue before making test on this module, and you will send me two e-mails. Now using Sys::Hostname instead of `hostname` and gracefully handling absolutely any combination of carriage-returns and line-feeds as valid line termination.
Wrapped all reads and writes to the SMTP socket in eval blocks, and installed a ALRM signal handler, for better handling of time-out conditions. Also added a $TipJar::MTA::TimeStampFrequency variable which is how many iterations of the main fork-and-send loop to make before logging a timestamp. The default is 200.
eval
minor cleanup.
AOL and Yahoo.com and who knows how many other sticklers require angle brackets around return addresses and recipients. Improved handling of MXes that we cannot connect to, by defining a ReQueue_unconnected entry point in addition to the ReQueue one that we had already..
ReQueue_unconnected
ReQueue
We now bounce mail to domains that ( have no MX records OR there is only one MX record and it is the same as the domain name ) AND we could not resolve the one name. Previously it had been given the full benefit of th doubt.
implemented domain listing for connection reuse. New messages and messages queued due to connection failure (but not 400 codes) get listed in the per-domain queue.
fixed a bug that caused the earlier of multiple messages handled in the same batch to get clobbered. Re-engineering domain file locking too.
we can now handle multi-line 250 responses. They exist. Also fixed a problem with retry deferral. And lowercasing of domain. And SMTP peers who give you a tiny packet on connection before they send their 220 greeting. And several "uninitialized value" warnings.
Adding a framework for remembering and caching recipient-based 4* and 5* errors for four hours.
the path to the dnsmx program is now configurable through a global variable $TipJar::MTA::dnsmxpath
$TipJar::MTA::dnsmxpath
The dependency on the dateheader package is now in the Makefile.PL
we now remember domains we have had trouble connecting to and wait at least $TipJar::MTA::ConRetryDelay seconds (defaults to seventeen minutes) after a failed connection attempt before trying again.
$TipJar::MTA::ConRetryDelay
We can handle peers that don't know what to do with a RSET command. It no longer shuts us down, rather we remember that the peer doesn't know how to reset its buffers and we prefer not to reuse a socket when sending them messages.
RSET
rewrote dnsmx() to use Net::DNS
added @NoBounceRegexList configuration variable, which is a list of sender-matching regexes that we don't bounce to -- very useful when you're running a C/R system and have a lot of bogus addresses you don't care to hear about
We no longer get confused by multi-line greetings, which may have been the source of many earlier confusions
log text changes
new conf variable $MaxActiveKids determines max parallel
fixed problem with zero-length message files
4xx error codes are now only cached $FourErrCacheLifetime seconds (defaults to 7 * 60 )
error code cache cleanup is fixed
now support 'nodns' use-line option to suppress loading Net::DNS and SMTProutes hash to provide hard-coding of mail exchange paths instead. And revised the licence.
moving newly appeared files from basedir into tempdir with unique names, instead of locking them, for all those situations where reading a file that was unlinked after it was opened just doesn't work.
the call to sleep($interval) is now checked and retried when it wakes up early due to the CHLD handler. Also there is new code to handle multiple recipients per SMTP transaction, which are organized by preferred MX and split into smaller groups when they are delayed at RCPT TO time. Yes this made things more complex, and it is not clear that it is bugless, so expect a 0.31 fairly soon.
sleep($interval)
Fixed a problem with requeueing messages with one recipient in 4XX situations. Added a new header to remember requeueing times
Two patches from RT queue, one quite old.
Thanks to Tim Esselens, the long-standing bug about error codes not actually expiring after four hours like they are supposed to has been repaired
Thanks to Jose Luis Martinez, The address to choose for the local end of sockets is now configurable through a global variable $TipJar::MTA::BindAddress, also a situation that resulted in false bounce-as-undeliverables has been mitigated.
$TipJar::MTA::BindAddress
my ... if 0 style statics have been refactored into outer scopes, and other style points causing warnings under 5.10 have been addressed as well.
my ... if 0
Thanks again JLMARTINEZ for bug #45702 and fix for it
We no longer send spaces after colons for compliance with RFC 5321 section 3.3.
we now have a TipJar::MTA::once subroutine that goes through the queue once then returns, also some modifications for working well on Windows
TipJar::MTA::once
Patches are welcome.
there is no rotation of the log in the mylog() function. mylog does reopen the file by name on every logging event, though. Rewriting mylog to use Unix::Syslog or Sys::Syslog would be cool, but would add dependencies. Mailing the log to the postmaster every once in a while is easy enough to do from cron.
mylog()
mylog
take advantage of post-RFC-821 features, specifically PIPELINING
use QMTP when available.
add MBOX and MailDir deliveries and some kind of configuration interface
see the README file for the license, which has changed. (you must submit your patches!)
David Nicol, <davidnico@cpan.org>
TipJar::MTA::queue.
To install TipJar::MTA, copy and paste the appropriate command in to your terminal.
cpanm
cpanm TipJar::MTA
CPAN shell
perl -MCPAN -e shell install TipJar::MTA
For more information on module installation, please visit the detailed CPAN module installation guide.