celmorlauren limited > Sendmail_M4.0.29a > Sendmail::M4::Utils

Download:
Sendmail_M4.0.29a.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Module Version: 0.27   Source  

NAME ^

Sendmail::M4::Utils - create and test sendmail M4 hack macro files

STATUS ^

Version 0.27 (Beta)

This compiles the M4 sendmail hack used by celmorlauren since version 0.23

HTML coding just STUBS at the moment.

SYNOPSIS ^

Sendmail is arguably the most powerfull and configurable e-mailing system in the world, however it does tend to be intimidating to System Adminstrators without a good foundation in programming. It is a very good idea to look at the "O'Reilly" publications "sendmail 3rd edition +" and their "Sendmail Cookbook", most tasks that need to be done can be solved by having a look at the "CookbooK".

Where a solution can not be found in the "Cookbook" or an existing "Hack" you will need to create your own.

Creating and testing sendmail hack macros can be a tiresome and error prone business, this script has been developed to help, however you will still need to understand sendmail macros to use this. Testing methods are desgined to be used by both the commamd line and via HTML using a web browser.

Please note that you will have to hand edit your sendmail m4 #.mc file, to include the reference to the hack being generated, below is an example taken from our own linux.mc file. The line you must include, begins with HACK the hack file follows, the current development version can be found as Sendmail::M4::Mail8 and Sendmail::M4::mail8, mail8 is the program, Mail8 is its module, see their documetation for more.

    dnl  We use the generic m4 macro definition. This defines
    dnl  an extented .forward and redirect mechanism.
    dnl
    DOMAIN(`generic')dnl
    dnl
    HACK(`mail8-stop-fake-mx')dnl
    dnl  These mailers are available. per default only smtp is used. You have
    dnl  to add entries to /etc/mail/mailertable to enable one of the other
    dnl  mailers.
    dnl
    MAILER(`local')dnl
    MAILER(`smtp')dnl
    MAILER(`procmail')dnl
    MAILER(`uucp')dnl
    MAILER(`bsmtp')dnl
    MAILER(`fido')dnl
    dnl
    dnl  Just an other (open)ldap feature is the usage of maill500 as mailer
    dnl  for a given (open)ldap domain (see manual page mail500).
    dnl
    dnl MAILER(`mail500', `place_here_your_openldap_domain')dnl
    dnl
    dnl  This line is required for formating the /etc/sendmail.cf
    dnl
    LOCAL_CONFIG

The most notable help are.

MACRO{

When constructing "macros" is the ability to "nest" called macros within the text block of the calling "macro", below is an example of the development version of our ANTI-SPAM hack.

 rule <<RULE;
 SScreen_Local_check_rcpt_1
 R $*.FOUND      $@ MACRO{ $1 # checking for localusers and Trouble Tickets
     R $*.mail3      $@ MACRO{ $1 # Trouble Ticket user
         R $&{CheckRcpt}     $@ MACRO{ $&{CheckRcpt} # Valid TT?
             dnl TT must conform to minimal rules
             R $*                $: $>Screen_Local_check_mail_2 $&{CheckHelo} 
         }MACRO
         R $*                $@ $>ScreenMail8blocker ${mail3tt}
     }MACRO
 }MACRO
 RULE

Without the "nested" macro structure this could be difficult to keep track of, and indeed it was, thats why we have developed this.

Inline MACRO

The above MACRO{ also handles INLINE MACROS which enable much used logical statements to be included without the cost of another rule-set, this module includes a selection of these.

Packed Macro {MashFound#}

Most of the included INLINE MACROS use the packed macro {MashFound#}, which are designed to hold 9 long-names each, which of the {MashFound#} macros being refered to is invisable to the developer|user. And during testing the normal macro statement {####} where #### is a macro contained by {MashFound#} may be used, the testing program does all the required conversions.

This is required due to the limited number of free to use long-names, sendmail assigns long-names for it-self at run-time. And so working OK during testing does not mean that sendmail will not fail at run-time. It is recommended to keep develeoper long-names to under 16.

TEST

Automated testing, the inclusion of test data within the source program, some of which is highly automated. It is very easy to generate 4000 lines of test results, the TEST setup has expected replys, so will only stop on the unexpected, so any changes to a script can be checked with ease.

After using this to generate your HACK M4 files you will never want to it by hand again!

This module is non OO, and exports the methods descriped under EXPORTS.

AUTHOR ^

Ian McNulty, celmorlauren limited (registered in England & Wales 5418604).

email <development@celmorlauren.com>

USES ^

IO::File

file creation

IPC::Open3

to start "sendmail -bt -Ctest.cf"

File::Copy

to copy "tee" file to "file" in sendmails "hack" directory.

English

Data::Dumper debuging this! used by our exported method "debug"

EXPORTS ^

HASH REF = setup(@_) returns HASH REF to internal hash %setup

This configures this module, and is always required first.

The %setup hash is enclosed in a BEGIN block, to ensure that all programs and modules that use this get the same settings.

Expected/Allowed values allways as a (hash value pairing).

hack_dir

SCALAR with default value of "/usr/share/sendmail/hack",

file

SCALAR "hack file name" to generate, with either full path or just the name, no default.

NOTE: "build" or "install" must also be specifed.

NOTE: if "install" is also defined a backup copy of "file" is made if it already exists!

sendmail

SCALAR with default value of "/usr/sbin/sendmail"

mc

SCALAR with default value of /etc/mail/linux.mc, this is the sendmail m4 source file to be used to build "cf", this is required for 'installation'

cf

SCALAR "test.cf file name" to build for testing purposes.

if "install" is specified and "cf" is not specified, will assume "test.cf" within current directory.

if "install" is specified and "cf" is is "sendmail.cf" will "die"!

otherwise will assume the main "sendmail.cf" is being tested.

html

HASH REF, default is 0

build

SCALAR Generate|build "tee" file, this does not require root permissions.

Enables you to check the "tee", before installing it.

NOTE: ignored if also "html".

install

SCALAR

SU "root" permissions are required. Copy "tee" file to "file", (sendmail hack directory file). Create "cf" file.

NOTE: ignored if also "html".

test

SCALAR Will "build"|"install" before "test" if specified.

silent

SCALAR

STOPS all output! AND character translation!! It is assumed that you are going to do something with the compiled rules.

error

ARRAY REF only when also "silent" has contents of "moan", "whoops" will allways simply exit.

UNKNOWN

ARRAY REF remaining unknown arguments supplied.

tee

SCALAR automatic info, name optained from "file", this file does not need "root" SU permissions, and is placed in the current working directory.

Installation phase copies this to "file" which will need SU perms!

NOTE: if "build" is also defined a backup copy of "tee" is made if it already exists!

log

SCALAR automatic info, as "tee" but appended with ".log".

This file is generated during non "html" testing, contains all data entered by yourself and from "sendmail -bt".

If "file" is not also defined then this file will not be generated.

testing

SCALAR automatic info, set when "test" starts, changes the way both "ok" and "echo" operate.

SU

SCALAR automatic info, is user "root":"root".

time

SCALAR automatic info, "time" script started.

macro

SCALAR automatic variable, incremented on MACRO statements

rules

ARRAY REF automatic list of read in "S" macro rules

rule

HASH REF automatic keyed by "rules"

Format

rule {

Stest_macro => {

S => []

contains complete "S" macro coding

H => []

HINT's as to use

O => []

keys for "T" in order of specification

T => {

TEST tests for coding

n => {

n = numeric count of test

see "rule::TEST" for details

}

}

M => []

contains list of SUB macros. TOP Level S only!

F =>

SCALAR only defined if FORCE is defined

N =>

SCALAR only defined if NOTEST is defined

G =>

SCALAR only defined if GLOBAL is defined

Top Level S only

1st line after S definition.

Reduces number of {macro_names} Limit of 96 !

}

}

inline

HASH REF automatic, where a rule is to be inlined, rule should start life as a standard rule above, when known to work OK, then inline. No other changes are needed. TEST lines etc are ignored.

Format is almost the the same as the the above rule, except most entrys are only here, so as not to break things.

Format

inline {

Stest_macro => {

S => []

contains complete "S" macro coding

G => SCALAR

only defined if GLOBAL is defined

I => []

contains list of sub inlines

H => []

exists only for compatability

O => []

exists only for compatability

T => {}

exists only for compatability

M => []

exists only for compatability

F => SCALAR

exists only for compatability

N => SCALAR

exists only for compatability

}

}

sane

HASH REF automatic, keyed normally by "rule", however anything may be used as a key.

Generated noramally by the method "sane" and refernced during testing by the MACRO TEST sub statement SANE "key".

Format

sane {

key => []

sendmail .D statements

}

testing_domains

HASH REF automatic, generated by method "testing_domains", used during testing.

Format

testing_domains {

OUR

[ HELO, DOMAIN, IP, RESOLVE, FROM, RCPT \n ],

OK

[ HELO, DOMAIN, IP, RESOLVE, FROM, RCPT \n ],

BAD

[ HELO, DOMAIN, IP, RESOLVE, FROM, RCPT \n ],

}

testing_domains_keys {

HELO => 0,

DOMAIN => 1,

IP => 2,

RESOLVE => 3,

FROM => 4,

RCPT => 5,

}

Lists|lines of "," delimited values.

"OUR" is your domain,

"OK" are legal domains and should be ok

"BAD" are faked|forged domains and should allways fail.

FOUND

HASH REF automatic, generated by MACRO statements such as FOUND, this uses just ONE "long name" to store as many FOUND statements as needed.

Format

FOUND => {

LIST => []

list of FOUND keys

KEY => {}

key is {macro} value is FOUND key

},

MASH_FOUND

HASH REF automatic, used during testing to keep current values for {MashFound} packed components.

Format

MASH_FOUND => {

macro => value,

macro => value,

macro => value,

macro => value,

}

magic

SCALAR special value used by this program, do not use.

paranoid

SCALAR value used by "Mail8" see its page for meaning.

debug @_

debug prints out caller info, and anything supplied to it, and asks for input, nothing and it will simply return, "n" or "no" and it exits.

Note any refs supplied will parsed by Dumper from package Data::Dumper

Included to help to debug this and modules that use it. Also when your code is OK it is easy to find and remove.

0 = moan(@_) allways returns 0

Either prints out to STDERR or to a <td><table> HTML table depending on use. Expects a list of moaning messages.

If setup{silent} places complaints in setup{error} instead of displaying

Perhaps this should be in Carp?

And just to let you know, our own comment module will be on CPAN soon, just as soon as the requested name space has been OKed, will be Carp::Comment, not uploaded yet due to the module that depends on it not being ready.

whoops(@_) allways exits

Based on moan and does much the same except it also exits.

Perhaps this should be in Carp?

$ok = ok("message") message defaults to "OK" or "TRY: "

NOTE:

NOT for HTML! or when "silent"

ALLWAYS does nothing, just returns 1 or 0 if "testing".

print "message?" allways apends a ?

Normal usage, when not "testing".

<STDIN> "reply" "y" or "CR"

returns 1 OK!

anything else

returns 0 NOT OK!

During "testing"

<STDIN> "CR"

returns 0

anything else

returned as is

@_ = translate @_

Does all the formating for echo & build.

Currently

UTF8 ("pound" UKP)|("euro" E) to $ conversion, also converts 3+ spaces to a tab.

EURO character works, but breaks Perldoc display for Perl 5.6! So for the pod bits EURO character is shown either as EURO or E.

POUND character works, but looks bad on CPAN, will display correctly on Perldoc for 5.8.8, but not on earlier versions, so is shown for these pages as POUND or UKP

echo @_

This produces output, both to the screen and to the "tee" file, most functions use this to output, this does a simple echo with no other formating other than shown below.

During testing no formating is done, text is output as is with just a "linefeed" appended.

Otherwise.

Sendmail expects tabed macro fields, however your "vi" session may be set to use spaces and colours etc, also "$" is used to signify a varity of things and this causes problems for Perl SCALARS.

To get round these problems, and to allow for better looking text.

In your code use at least 3 spaces where sendmail expects a "tab", and use ("POUND" or "EURO") where sendmail expects a "$", however if you are not using a keyboard with either of these symbols then you will have to escape \$ as normal.

"echo" does UTF8 ("pound" UKP)|("euro" E) to $ conversion, also converts 3+ spaces to a tab, this is done via translate above.

NOTE:

NOT for HTML! or when "silent"

ALLWAYS does nothing, just returns 1 or 0 if "testing".

dnl @_

For sendmail "dnl" comments, wraps supplied args in "dnl".

NOTE:

NOT for HTML! or when "silent"

ALLWAYS does nothing, just returns 1 or 0 if "testing".

define_MashFound @_

It is safest to define {MashFound} before use, supply it with a list of {macro names} which will be stored within this packed macro, sets up %setup{FOUND} and %setup{MASH_FOUND}.

testing_domains @_

"testing_domains" expects at least two arguments|lines, the first is the key for the HASH setup{testing_domains}, remaining argument|lines are ("," delimeted (HELO, DOMAIN, IP, RESOLVE, FROM, RCPT) values, which are for use during testing.

Referenced during testing by TEST AUTO(key KEY sub_key1 sub_key2,)

where key is one of (E,F,O,V), KEY is one of (OUR,OK,BAD), and sub_key# is one of (HELO,DOMAIN,IP,RESOLVE,FROM,RCPT)

Format

    OUR
    mail.celmorlauren.com, 0, 80.176.153.184, FAIL, development@celmorlauren.com, ian@daisymoo.com
    mail.celmorlauren.co.uk, 0, 80.176.153.184, FAIL, development@celmorlauren.co.uk, ian@daisymoo.com
    mail.daisymoo.com, 0, 80.176.153.184, FAIL, development@daisymoo.com, ian@daisymoo.com


    BAD
    this.is.bogus.bogus, 0, 10.0.3.4, FAIL, you@localhost, ian@daisymoo.com

So long as there is a blank line, between keys then definitions for OUR,OK,BAD can be done together as the sample above shows. This also allows "#" comment lines to be included for clarity.

You may notice that our IP does not resolve to a domain, that is a common problem and so Mail8 does not care about that, it only cares that the HELO resolves to the connected IP, the RESOLVE of OK stops a DNS look-up.

SCALAR inline SCALAR (optional)

single argument must be either 0 or <1> or someother scalar quantity. Always returns current value for inline. Argument is purely optional,if not supplied just returns current value.

This switches ON or OFF the INLINE statement for rules and MACROs contained within them, enabling inline cabable rules to be tested as seperate macros and then inlined when known to be OK, it should be noted testing is required to ensure the inlining does not cause unwanted side effects.

Initial value is OFF|0

sane @_

"sane" expects at least two arguments|lines, the first is the key for the HASH setup{sane}, remaining argument|lines are statements to be encoded as sendmail -bt .D statments, statements are "," delimited.

Referenced during testing by TEST SANE(key)

Format

    Local_check_mail
    {client_addr}127.0.0.1, {client_name}Localhost, {client_resolve}OK

rule @_

"rule" is the main worker, sendmail macros are very powerfull and usefull, you will need to understand the "sendmail" macro programming syntax to use this.

  1. 1st argument|line is the "S" macro rule, which must start with "S".
  2. 2nd argument|line GLOBAL A were A is the letter to use. OPTIONAL

    GLOBAL is a special argument that is used to reduce the number of sendmail {macro_names}, as sendmail has a limit of 96. It works by using the letter specified (defaults to Z) to base its naming policy, sub macros are numbered from ZERO. Use it if you have the sendmail error message "too many long names"

  3. 2nd or 3rd argument|line INLINE code is intended to be (inlined)

    INLINE is a special argument that is used to reduce the number of sendmail "named rulesets" as sendmail has a standard limit of 100. Used with the method inline this will inline code rather than define them as rule sets, resulting in a lower count of rule sets at the expense of larger file size. Use it if you have the sendmail error message "too many named rulesets".

    Best policy is to test small sections as "rule sets" and inline when noted to be OK. But remember to ensure all works OK when inlined.

Remaining argumentslines are the Macro, normally starting with "R", or something that make sense as a macro to sendmail. The generated macro code returns the supplied arg by default, unless the code returns first.

Extensions to the sendmail syntax are

INLINE

INLINE must be the very first line, (after GLOBAL if used), this inlines this macro rule instead of producing a real named ruleset, this statement only has effect if inline 1 has been used, otherwise it only modifies the generated maco not to return the original saved value (so as not to break things when inlined).

INLINE supports sub arguments

ALLWAYS which overrides the global value of $inline, meaning that this code will allways be INLINED, also that this code is expected to allways work correctly and does not require any testing, please refrain from using this youself as it is intended for internal program use. Most if not all internal MACROS are coded this way.

Note: ALLWAYS is the 1st sub argument after INLINE, and other sub arguments may follow.

    Usage:
            INLINE ALLWAYS MASH
            INLINE ALLWAYS MASH TempA

NOMASH which also stops the normal action of saving the original value.

    Usage:
            INLINE NOMASH

MASH retores original saved value at the end of this macro rule, so for routines that are much used, they remain more like the original MACRO specification (without INLINE), also a over-ride value for MASH may follow, internal methods use TempA which results in {MashTempA}

    Uasage:
            INLINE MASH
            INLINE MASH TempA

If a named rule seet is inlined all its component MACROs must also inlined! and so must also be compliant with INLINE usage.

Also note it is advised that GLOBAL has also been specified, otherwise this will assume the default GLOBAL of Z.

Note all code within the INLINED macro must be compliant with the usage, use of a RHS $@ will cause this to whoops complaing about the infrigment of use.

Otherwise all the things that a normall macro use may be specified, however when inline is in effect all TEST lines are ignored.

May be used in explicitly named rulesets and MACROs, the entire line R $* $: $ruleset $1> is replaced with the inlined code that the ruleset refers to.

OPTION

OPTION must be the very first line, (after GLOBAL if used), and can not be used with INLINE, it supports sub arguments that alter the formatation of normal non INLINE macros.

OPTION supports sub arguments

NOMASH which also stops the normal action of saving the original value.

    Usage:
            OPTION NOMASH

MASH which forces the Macro to use a known value for its mash

    Usage:
            OPTION MASH 1

    Which generates {MashA1} if GLOBAL is A            

# comment line within Rule to improve readability, otherwise ignored

MACRO MACRO{ }MACRO

$: MACRO{ $1 # comment == $: $>Sub_something $1 comment

MACRO{ opens a block, }MACRO terminates the block.

Enables a sub macro that is used only once to be contained within the calling macro stament block, it is however coded in the normal way in the hack file. MACROs may be nested as deeply as required, enabling easy to code and read complex IF|ELSE statment blocks. Example below.

 rule <<RULE;
 SSome_macro
 R $*.FOUND      $@ MACRO{ $1 # something.FOUND
     R $*.mail3      $@ MACRO{ $1 # something.mail3.FOUND
         R $&{CheckRcpt}     $@ MACRO{ $&{CheckRcpt} # Valid TT?
             dnl TT must conform to minimal rules
             R $*                $: $>Standard_TT_mail $1
         }MACRO
         R $*                $@ $>SBad_mail $1
     }MACRO
 }MACRO
 RULE

Please do not use the macro named SScreen_macro yourself as it is used by this method appended with numerics

DEFINE_MASHFOUND

Must be used after the Perl statement define_MashFound and before any M4 macro statements that refer to the packed macro {MashFound}.

This should be placed in the first rule that is used, and before any other capatalised macros, such as FIND IS etc. Failure to do so will cause unpredictable errors elsewhere when running the M4 hack file.

FOUND

expects a single argument, which is the {macro} to be loaded with the $+.FOUND if that is the case, this is a an inbuilt INLINE ALLWAYS MACRO which generates code to be included in m4 source.

Usage:

    FOUND BadRelay

BadRelay will be loaded with $+.FOUND only if R $+.FOUND, current work space is saved and restored.

comments may be used, this will be included as a "dnl" line within the macro

It should be noted that only {MashFound} is used, the {macro} is now a key to an internal array kept by {MashFound}, this compexity is required due to the limited number of "long names" available to the developer, testing does not show up these limitations, it requires sendmail to be run for real and observed while talking to other servers.

FIND

expects a single argument, which is the {MashFound}-{macro}> to be accessed and have its contents placed in the workspace, this is now the only way to access items saved by FOUND.

this is a an inbuilt INLINE ALLWAYS MACRO which generates code to be included in m4 source.

Usage:

    FIND BadRelay

STORE

expects a single argument, works like FOUND excecpt allways loads value with current work space.

this is a an inbuilt INLINE ALLWAYS MACRO which generates code to be included in m4 source.

Usage:

    STORE BadRelay

IS

Expects upto 3 arguments. Number expected depends on the first argument.

FOUND expects 2 sub arguments.

  1. is the {macro} to check for .FOUND, just the name, do not enclose in brackets.
        IS FOUND Bounce
  2. is the action to do if .FOUND, since the nature of this INLINE ALLWAYS MASH macro never varys the normal form would be $@ $>SomethingOrOther $1 alternativly if you do not care about the returned value $: $>SomethingOrOther $1 or even $#err something
        IS FOUND Bounce $# "Bounce not wanted here"

THISFOUND expects 1 argument, the action as FOUND

checks current work space for .FOUND

    IS THISFOUND $@ $1.FOUND

REFUSED and ALREADYREFUSED expects 1 argument, the action as FOUND

Normally the action should be #err somthing

REFUSED and ALREADYREFUSED the checked {macro} is either {Refused} or {AlreadyRefused}, these macro's are used by Mail8, however we feel that these are usefull to other scripts.

AND (REFUSED|ALREADYREFUSED) $#err somthing

AND is a special sub macro statement that allows the actions that REFUSED|ALREADYREFUSED does to be enacted also without the cost of another rule set. See below, we are not refering to the "IS REFUSED"!

    IS FOUND Bounce AND REFUSED  $#err somthing

REFUSED and ALREADYREFUSED

These INLINE ALLWAYS MASH macros, load the {client_addr}.FOUND into the {macro} which is either {Refused} or {AlreadyRefused}, a single sub argument is expected, which is the action to do, however if the sub argument is ommited, this will simply store and do nothing else.

Normally REFUSED $#err something

{MashSelf}

{MashSelf} provides access to the autosaved argument for this rule.

Usage R $* £: &${MashSelf}

{MashStack}

{MashStack} provides a lasy way to keep data, without polluting other data. Allways append something to the "MashStack", such as "A" as shown in the example.

Usage R $* $: &${MashStackA} R $* $: &${MashStackB}

{MashTemp}

{MashTemp} provides a lasy way to keep very temporary data, these values are only dependable within the current Macro, and may be clobbered by contained Macro's. This method exits to reduce further the number of sendmail {macro names}. Allways append something to the "MashTemp", such as "A" as shown in the example. Remember to use a consistant sub naming policy to minimise the generated names, we recomend using the sequence (A,B,C,D ..) but use as few as possible.

Usage R $* $: &${MashTempA} R $* $: &${MashTempB}

DEBUG switchs on|off debug info during read-in

Errors in the macro TEST coding can be difficult to track, so this will display helpfull debuging info, remove when the problem has been sorted.

Usage DEBUG 1 To switch on DEBUG 0 To switch off DEBUG To switch off, however its best to be explicit.

TEST

TEST macro code, is for testing of the macro, this code does not enter the output file.

TEST lines are converted into a simple HASH as follows

{

D => []

list of .D define a Macro statements

T => SCALAR

translation macro, to be used before values below are supplied to the macro under test

V => []

values to try with macro

E => []

values as "V" but must result in "ERR"

O => []

values as "V" but must result in "OK"

F => []

values as "V" but must result in "FOUND"

I# => []

values as "V" but must result in "#"

where "#" is the expected reply.

eg

IREPLY

SANE => []

list of $setup{sane} keys that define lists of .D define a Macro statements

AUTO

does not have a HASH, but instead creates (V,E,O,F) as required.

}

Encoded with leading definition letter and opening bracket, values "," delimited. D() D( {client_addr}198.168.2.1, {client_name}dog.bone.com ) T() T(Translate) V() V(frodo\@hobit.com, frog\@pond.com)

Not all definitions are required, you may use all or just one, in the case where no enclosing "()" brackets are used, this assumes you mean "V()". E and O will stop|interrupt testing if returned result is unexpected. V will stop|interrupt testing if result is either "ERR" or "OK"!

Examples below

    TEST SANE(std) D({client_addr}198.168.2.1, sdog.bone.com) V(frodo\@hobit.com) 

Assumed "V()" values for macro

    TEST frodo\@hobit.com, frog\@pond.com 

Testing "Local_check_relay" requires "host.name"$|"ip_address", which requires our build "Translate" macro or your own for other uses.

    TEST T(Translate) E(bogus.host.domain 12.5.7.89, n.n.bogus 1.2.3.4)

TEST methods are used in order of specification, and effects persist during testing, so things defined for a preceding "Macro" will effect all "Macros" that follow

AUTO

AUTO enables local site checking, without the need to hack the module, or expect module methods to modify the TESTS from their command line, set this up with the method testing_domains, do not use other TEST methods with this apart from SANE, T and (D where AUTO D is not used).

General format for this is (except for D)

AUTO(key KEY sub_key1 sub_key2, key KEY sub_key1 sub_key2, ...

Where key is one of (E,F,O,V), KEY is one of (OUR,OK,BAD) and sub_key# is one of (HELO,DOMAIN,IP,RESOLVE,FROM,RCPT).

Foreach setup{testing_domains}->{KEY}->[] line, the relevent field is used for testing, and so has the effect of specifying TEST E(...........) where each "." is the relevent field referenced by sub_key#.

D

AUTO(D; KEY; M sub_key1; M sub_key1; M sub_key1, ...

Where KEY is one of (OUR,OK,BAD), M is a sendmail macro name, enclosed in {} if that would normally be required, and may be anything that can be defined, sub_key1 as already defined.

Please note the use of ";" to delimit fields, do not forget to place a ";" after the D and the KEY even if you are only defining a single macro.

This is not of any use without other TEST options, being specified. If used D generates a TEST line based on the other TEST options for each setup{testing_domains}->{KEY}->[] line. And so has the effect of specifying.

    TEST D({macro}value,{macro}value) E(...........) V(.......)
    TEST D({macro}value,{macro}value) E(...........) V(.......)
    TEST D({macro}value,{macro}value) E(...........) V(.......)
    TEST D({macro}value,{macro}value) E(...........) V(.......)
    ......

HINT

HINT is used to supply hints during testing, examples as to expected format etc, use as many as required, or none at all, but it will make your life easier to use them if you do not include TEST code or want to enter data on the fly.

All HINT are stored in the H=>[] ARRAY for the rule

Example below

    TEST D({client_addr}198.168.2.1, sdog.bone.com) V(frodo\@hobit.com) 
    HINT email address expected, valid or invalid

FORCE

FORCE if specified will allways pause testing and ask you for test data, regardless of wether TEST has been used, has no meaning for "HTML", and omitting TESTs has the same effect. Some sort of hint should follow, which will be shown before asking you for data.

FORCE is stored in the F=>SCALAR for the rule

Example below

    TEST D({client_addr}198.168.2.1, sdog.bone.com) V(frodo\@hobit.com) 
    FORCE email address expected, valid or invalid

NOTEST

NOTEST if specified is the reverse of FORCE, meaning if no TESTs have been defined, this will allways skip testing, and continue. Some sort of hint should follow, explaining why testing is not required.

If NOTEST AUTO is specified then it is assumed that the code is program generated and is tested by a controlling macro, so this will stay quite about it, otherwise this will moan about the lack of testing.

Note if both FORCE and NOTEST are defined, NOTEST takes precedence.

NOTEST is stored in the N=>SCALAR for the rule

Example below

    NOTEST containing rule tests this.

inbuilt_rule @_

Enables this to test sendmails own internal rules, instruction format is the same as for the above rule, indeed this uses the same %setup HASHs.

NOTE: This only supports the test methods, even though it uses the same macro parser to its work, nothing is output, and the "S", "M" and"N" componants are removed for safty reasons, and a "I" with the value 1 is added.

VERSIONID $title

Only argument expected is the title|name for this hack to insert in the VERSIONID statement. Output format is.

    # version
    my ($title) = @_;
    my $time = localtime();
    echo "VERSIONID(`@(#)$title for Sendmail 8.12 or better $time')";

LOCAL_CONFIG

Required statement, this inserts required statments into the hack file.

    echo <<ECHO;
    LOCAL_CONFIG
    KSelfMacro macro
    ECHO

Currently only the SelfMacro macro, which is used by many of the above methods, feel free to use it yourself but do not use names starting with Mash other than those stated in rule above.

Add your own definitions after this.

LOCAL_RULESETS

Required statement, this inserts required statments into the hack file. Currently only a Translate macro, which is based on the example in the Sendmail 3rd edition book, section 7.1.1, page 290, however we will assume only 2 tokens are going to be supplied (the program inserts the seperator), this is for the standard macro Local_check_relay

Due to the limited number of "long names", some have had to be recoded as an $| delimited array {MashFound}, which of course makes testing difficult, so as we already have a problem with "rule sets", "Translate" will now also pack {MashFound}, which is re-writen each time this is used.

    echo <<ECHO;
    LOCAL_RULESETS

    STranslate
    R $* $$| $*     $: $1 $| $2     fake for -bt mode
    ECHO

Add your own definitions after this.

build

No arguments, this may included in the script after the rules and just before install, this has no effect unless setup{silent} is in effect, meaning that preceeding rules have not produced output, or you have built the required setup HASH yourself.

install

No arguments, this may be included in the script after the rules or build and just before test, if you are not root this will attempt to su -c '"program" install 1'

Note you may call your program with "install 1" so long as setup processes the program arguments, or at least gets 1st pick. You will have to ensure that setup gets all its requires.

Testing methods ============================ ^

Sendmail intialization and chit chat methods, usable directly. But normally used by test specified further down this document.

REF HASH setup{senddmail_hash} = sendmail_hash

Setup script for sendmail below, call it yourself to get the "setup" that will be used by sendmail, mostly of use to initialize the output methods with something more suitable for your needs, this currently defaults to methods suitable for command line usage.

If used place before test to enable your alternative setup, otherwise omit and use the default settings. If you use this directly be sure to also use sendmail with no arguments to intialise the connection, sendmail -bt gives a greating message on starting.

NOTE calling it replaces the existing HASH with the default.

sendmail calls this itself if the required HASH does not exist!

    sendmail_hash => {
        IO  =>  {   IO::File objects used by IPC::Open3 open3 
            r    => IO::File object
            w    => IO::FIle object
            e    => IO::File object
            pid  => IPC::Open3 open3 object 'sendmail'
        }
        select  {   IO:Select objects which refer to above IO::File objects
            r   =>  IO::Select object
            w   =>  IO::Select object   timeout has 30 seconds added to it
            e   =>  IO::Select object
            t   =>  SCALAR = 3  timeout seconds for select statment
            l   =>  SCALAR      last action that caused this to return 
                                one of 
                                r=(read),w=(write),e=(error),t=(timeout)
        }
        buffer  {   [] REFs containing data for|from above IO::File objects
            r   =>  [] REF  contains read in data (push)
            w   =>  [] REF  contains data waiting to be written (shift)
            e   =>  [] REF  contains errors (push)
            l   =>  [] REF  contains last read in data or error
        }
        error   =>  [] REF  general errors, undef if OK
        output  {   what is this supposed to do with 'display' infomation?
            silent  => SCALAR = 0   1 suppresses all output
            echo    => SUB REF default is &echo (command line only)
            moan    => SUB REF default is &moan 
                                       (which already understands HTML)
            whoops  => SEB REF default is &whoops 
                                        (based on moan, but also exits)
        }

undef sendmail_whoops @_

sendmail methods use this to complain and exit, will be silent if sendmail_hash-output->silent>, alternativly uses the relevant whoops method to complain and exit. NOTE will allways exit.

undef sendmail_moan @_

sendmail methods use this to complain and to fill out its own sendmail_hash{error}, will be silent if sendmail_hash-output->silent>, alternativly uses the relevant moan method to complain.

undef sendmail_echo @_

sendmail methods use this to display the output of "sendmail -bt" interprocess pipe, will be silent if sendmail_hash-output->silent>, alternativly uses the relevant echo method to display.

($code,@buffer) = sendmail(@_)

Interface for talking to "sendmail -bt", on first call will set it self up using sendmail_hash if the required HASH does not already exist.

Any arguments are "sendmail instructions" this will allways append newlines.

Returns recieved @buffer, does not return on writes as sendmail will allways reply, however returns undef on timeouts or on read and write fails!

sendmail has its own "sendmail_hash" HASH in setup, which will be setup on first use if not already defined, and enougth other information exists to enable this.

USES

sendmail_whoops to complain about errors and exit! sendmail_moan to complain about errors! sendmail_echo to display received data

test @_

Expects either

nothing, in which case all defined rules are tested in turn, if any "rule" does not have "TEST"s defined for it, this will halt on and ask you for a test value, or simply press return to continue, HTML format is still in development.

rule=>test, rule=>test, rule=>test hash value pairs, which are the rule to test and the TEST number to do, or alternativly the word "ALL" to do all "TESTS" for this rule.

This will only "TEST" rules that have been defined, so it is best to place this last in your code. This uses sendmail to talk to "sendmail -bt" via open3.

sets setup{testing} to inform other methods that are common to both build and test to use setup{log} instead of setup{tee}.

Example USAGE from a command line driven program ^

Note this also contains a cut down snippet of the ANTI SPAM hack that caused this to come into existance.

    #! /usr/bin/perl -w
    use Sendmail::M4::Utils;

    setup @ARGV;

    # copyright message
    dnl <<DNL;
    Copyright (c) 2007 celmorlauren Limited England
    Author: Ian McNulty       <development\@celmorlauren.com>

    this should live in /usr/share/sendmail/hack/mail8-stop-fake-mx.m4

    some settings that are advised
    FEATURE(`access_db',        `hash -T<TMPF> -o /etc/mail/access.db')
    FEATURE(`greet_pause',      `2000')
    define(`confPRIVACY_FLAGS', `goaway')
    DNL

    # version
    VERSIONID "ANTI SPAM";

    # 
    dnl <<DNL;

    SPAM checking additions --------------------------
    '-' added to trap DSL faked domain names

    DNL
    echo <<ECHO;
    define(`confOPERATORS',`.:@!^/[]-')
    ECHO

    LOCAL_CONFIG

    echo <<ECHO;
    KRlookup dns -RA -a.FOUND -d5s -r4

    ECHO

    # we can do some checking with HEADER lines
    echo "HReceived: $>+ScreenReceived";


    ################################################################
    ################################################################
    # end of snippet, this would of course contain your own code
    ################################################################
    ################################################################

    # this is the start of the real code
    LOCAL_RULESETS

    echo <<ECHO;
    dnl this bit is for mail8, intial contact and flood checking?
    dnl bit below checked, see p288
    ECHO

    #######################################
    # CONTACT
    # This bit arrived at on first contact, and so permissions based on IP can be set
    rule <<RULE;
    SLocal_check_relay
    TEST T(Translate) V(local 192.168.0.1, bogus.host 1.2.3.4)
    R $* $| $*      $: $(SelfMacro {RelayName} $@ $1 $) $1 $| $2
    R $* $| $*      $: $(SelfMacro {RelayIP} $@ $2 $) $1 $| $2
    R $*            $: $>Screen_bad_relay $&{RelayIP} 
    RULE

    intstall;

    test;
    
    ################################################################
    ################################################################
    # end of snippet, this would of course contain your own code
    ################################################################
    ################################################################

HISTORY ^

Versions

0.1

Nov 2006 1st version, pure sendmail M4 hack, using plug-in Perl programs.

0.2

25 Aug 2007, this 1st CPAN test module, developed to test M4 hack scripts, original script split into Utils for creation and testing, and Mail8 the ANTI SPAM engine.

Amendments to release version

30 Aug 2007

TEST, HINT & FORCE did not nest.

3 Sept 2007

cf file backup now has a tilde ending "~". %setup{paranoid} added for mail8.

5 Sept 2007

NOTEST, for nested MACROS that are already tested by a containing level, or where additional testing makes no sense.

8 Sept 2007

Testing of a Mail8 component with bugs caused files with wrong permisions to be created, meaning the standard user could not re-create them, and some confusion as to what was happening. Utils will now whoops on these problems giving a clear indication as to the real problem.

{MashStack} failed to work when more than one instance was used on a single line.

NOTEST AUTO will not moan meaning auto generated lines that not be meaningfully tested do not complain about it.

FORCE and absence of TEST's now will continue to ask for input for a rule, until nothing is entered

10 Sept 2007

Testing of Reintergrated Mail8 showed that NESTing still did not work, reason found and fixed, also somethings that where expected were not allways supplied.

GLOBAL added to reduce the number of {macro_names} as Mail8 managed to go over sendmails limit of 96, used at the top level S rule to reset counters.

11 Sept 2007

INLINE added, Mail8 managed to go over the standard sendmail limit of 100 named rulesets, counted a total of 123 in the test.cf, we know we could re-compile sendmail with a bigger limit. But that is something we can not expect of anyone else.

13 Sept 2007

UTF8 EURO currency "character" added can now be used in rule definitions, where $ would have to be escaped.

14 Sept 2007

FOUND inbuilt MACRO added to load SelfMacro {macro} with "$+.FOUND", intention is to remove another rule set as this MACRO will be coded INLINE.

15 Sept 2007

method inbuilt_rule added to enable testing of sendmails own rule sets, these use the same methods and control HASHs as rule except generates no code.

16 Sept 2007

MACRO{ statements (REFUSED, ALREADYREFUSED, IS (REFUSED, ALREADYREFUSED, FOUND), INLINE ALLWAYS) added to both help with reducing the number of generated rule sets and to improve the layout of Mail8.

17 Sept 2007

MACRO{ TEST sub statement SANE and the method "sane" added to simplify reseting sendmail -bt test session to sensible values.

19 Sept 2007

MACRO{ TEST sub statement AUTO and method "testing_domains" added to enable customers vary the test data to reflect their setup, testing Sendmail::M4::Mail8 via Sendmail::M4::mail8 with just celmorlauren email setup is not sufficient.

21 Sept 2007

Documentation clean up, noted that EURO character causes problems with Perldoc for version 5.6 Perl, POUND does not work either (but at least does mess up display)

0.21

21 Sept 2007 CPAN Amended version

Amendments to release version

22 Sept 2007

Documentation clean up, noted that POUND character does not display correctly on CPAN, hum it would be better if CPAN coped with UTF8 characters!

MACRO{ DEBUG statement added to switch on debuging within the TEST line read in phase, to track difficult to see errors.

{MashSelf} failed to work when more than one instance was used on a single line.

0.22

22 Sept 2007 CPAN Amended version

Amendments to release version

22 Sept 2007

installed on a test system, started to run ("too many long names" again) AAARGH!

{MashTemp} added a variant of {MashStack}, differnce being the reduced number of names generated, the names only being safe only in the current macro, and can be clobbered by contained macros, that use this. You have been warned!

23 Sept 2007

Macro{ OPTION added, this is to enable such things

OPTION NO MASH

OPTION MASH 1 mash nameing policy uses {Mash1}

Also added sub option to INLINE ALLWAYS MASH, which overides the normal macro nameing policy, internal methods now use the mash name {MashTempA} for purposes of saving and returning a value.

0.23

23 Sept 2007 CPAN Amended version

Amendments to release version

01 Oct 2007

Live on primary, secondary and test systems. When sending mail to "sendmail.org" (via test), sendmail tried and failed to allocate more names for itself ("too many long names" again) AAARGH! However the send did still work (without md5)

Currently the Sendmail::M4::Mail8.pm version uses 21 "long names", OK for normall sending. But md5 needs more.

  • MACRO{ statement FOUND & IS FOUND modified, new statements FIND & STORE, now a single {MashFound} macro can be loaded with as many sub names as required.
  • Translate rule set modified to pack {MashFound}
  • define_MashFoud added, to declare packed components of {MashFound}

    Modifications made so that {macro} maybe used for TEST D & SANE statments, but will be packed into {MashFound} if they have been defined.

0.24

22 September 2007 CPAN Amended version

Amendments to release version

08 Oct 2007

Error in pod line 960 space between =head 2, as Mail8 has been updated with Reply-to header line checking, this little thing can be fixed and uploaded.

0.25

08 October 2007 CPAN Amended version

Amendments to release version

12 Oct 2007

FIND did not have NOMASH stated, so used a long-name when it should not have had done.

Mail8 development, dealing with hotmail & yahoo mail addresses and domains, showed that sendmail has a problem with wild-cards higher than $9, use a $10 and sendmail will complain ( too many wildcards ).

define_MashFound ammended along with others that use the packed form of {MashFound}, although sendmail has a limit of 9 wildcards, {MashFound#} where # is numeric, each containing upto 9 elements, the presence of these makes no other difference to the macro coding, FIND STORE etc all work as before.

13 Oct 2007

POD clean up, HISTORY moved to end of document, layout of POD improved, but some bits will be left for later. Code check. Should not touching again until the socks method is ready.

0.26

13 October 2007 CPAN Amended version

Amendments to release version

14 Oct 2007

Mail8 added another component to MashFound# making 9 in one, causing M4 statement not to formated correctly, failure in logic fixed.

0.27

14 October 2007 CPAN Amended version

Amendments to release version

syntax highlighting: