Net::Dev::Tools::Syslog - Send, Listen Parse Syslog messages.
Syslog 1.0.0
use Syslog; # # Syslog Parser # ($syslog, $error) = Syslog->parse( -dump => <directory>, -append => <0|1>, -ext => <extension>, -report => <0|1>, -interval => <seconds>, -debug => <0|1|2|3>, -rx_time => <0|1>, -lastmsg => <0|1>, -min_date => <mm/dd/yyyy [hh:mm]>, -max_date => <mm/dd/yyyy [hh:mm]>, -device => <pattern>, -tag => <pattern>, -message => <pattern>, -format => <bsd|noHost|self> -moreTime => <0|1> -parseTag => <0|1> ); $parse = $syslog->parse_syslog_line(<line>); # # Syslog Send # ($send, $error) = Syslog->send( -server => <address>, -port => <IP port>, -proto => <udp|tcp>, -facility => <facility>, -severity => <severity>, -timestamp => <timestamp>, -device => <device name>, -tag => <tag>, -pid => <pid>, -message => <message>, -strict => <0|1>, ); $send->send_message( -server => <address>, -port => <IP port>, -proto => <udp|tcp>, -facility => <facility>, -severity => <severity>, -timestamp => <timestamp>, -device => <device name>, -tag => <tag>, -pid => <pid>, -message => <message>, -strict => <0|1>, ); # # Syslog Listen # ($listen, $error) = Syslog->listen( -port => <IP port>, -proto => <udp|tcp>, -maxlength => <integer> -verbose => <0|1|2|3>, -fwd_server => <IP address>, -fwd_port => <integer>, -fwd_proto => <udp|tcp>, );
Module provides methods to parse syslog files, send syslog messages to syslog server, listen for syslog message on localhost.
parse method creates a class that configures the parser used on each syslog file entry (line) sent to the parser. The object is first created with properties that define how a syslog line is to be worked on. The parse_syslog_line function (method) is then used to parse the syslog line and return a reference to a hash.
send method will send a syslog message to a syslog sever. The user can provide as much or as little information desired. The class will then create a syslog message from the information given or from default values and send the message to the desired server.
listen will open the desired port on the local host to listen for sylog messages. Message received on the port are assumed to be syslog messages and are printed to STDOUT. Messages can also be forwarded 'as received' to another address. The functionality of the parser can also be used when in this mode.
See documentation for individual function/methods for more detail on usage and operation.
Time::Local used to convert TIMESTAMP to epoch IO::Socket used by send and listen methods Sys::Hostname used to support DNS
parse_syslog_msg epoch_to_syslog_timestamp make_timeslots epoch_timeslot_index normalize_facility normalize_severity
:parser parse_syslog_msg :time epoch_to_syslog_timestamp, make_timeslots, epoch_timeslot_index :syslog normalize_facility, normalize_severity
Constructor to create an object to parse the lines of a syslog file. Arguments are used to define parsing. See function parse_syslog_line section to see how to parse the line.
($syslog, $error) = Syslog->parse( -dump => <directory>, -append => <0|1>, -ext => <extension>, -report => <0|1>, -interval => <seconds>, -debug => <0|1|2|3>, -rx_time => <0|1>, -lastmsg => <0|1>, -min_date => <mm/dd/yyyy [hh:mm]>, -max_date => <mm/dd/yyyy [hh:mm]>, -device => <pattern>, -tag => <pattern>, -message => <pattern>, -format => <bsd|noHost|self>, -moreTime => <0|1>, -parseTag => <0|1>, );
Argument Checks:
If -dump is used, then argument must be a directory, current directory is not assumed. The directory provided must exist and allow user write access.
If -interval is less than 60, it is set to 60.
If -min_date and/or -max_date are given the syntax and range are checked.
Return, in list context will return reference to object and error. In scalar context returns reference to object. If object fails to create, then the reference is undef.
Enable creation of separate syslog files. Each file created will only contain lines for the device defined in the syslog message HOSTNAME. The <directory> argument defines a directory to where device specific syslog files are dumped. Current directory is not assumed. Directories are checked for existence and writability. Default = FALSE, no dumps
If 0, device files created due to -dump are overwritten. If 1, device files created due to -dump are appended to. Default = 0, (overwrite)
File extension to use for device files created due to -dump being enabled. Default = 'slp', (SysLog Parsed)
If 0 no stats are recorded. If 1 stats are extracted. For each line successfully parsed, information is stored in a hash. This hash can be referenced with the function syslog_stats_href.
The amount of seconds to use when making timeslots. make_timeslots function will make timeslot ranging from min and max time found or given. The timeslot info can then be used to create stats for desired time intervals. See @TIMESLOTS for more info. Min value is 60 (1 minute). Default is 3600 (1 hour).
Set debug level, verbosity increases as number value increases.
Set flag to use the localhost receive time and not the timestamp from the sylog message. Some syslog deamon prepend information to the syslog message when writing to a file. If a receive time is one of these fields, then it can be used. This will normalize all times to when they are received by the serever. Default is 0
Set flag to to handle last message as previous message. If true and the syslog message has the pattern 'last message repeated <N> time',then we replace this current line with the previous line. Otherwise the 'last message' line is treated as all other syslog lines. The tag will be defined as 'lastmsg', pid and content set to ''. Default is 0.
If given, then will be used to filter dates. Only lines with dates greater to or equal to this will be be parsed. This check is performed after -rx_time, thus filter applies to whatever date you decide to keep. You must enter mm/dd/yyyy, other values will default: ss defaults to 0 if hh:mm given, 59 if no time given mm defaults to 0 if hh: given, 59 if no time given hh defaults to 23 if no time given Mmm/dd/yyyy can also be use, where Mmm is Jan, Feb, Mar,...
If given, then will be used to filter dates. Only lines with dates less than or equal to this will be be parsed. This check is performed after -rx_time, thus filter applies to whatever date you decide to keep. Apply same syntax rules as -min_date
If given, only device fields matching the pattern are kept. Text strings or Perl regexp can be given.
If given, only tag fields matching the pattern are kept. Text strings or Perl regexp can be given.
If given, only message fields matching the pattern are kept. Text strings or Perl regexp can be given.
Defines how $SYSLOG_pattern is constructed to use match the syslog line against to parse out the component parts. bsd will use $TIMESTAMP, $HOSTNAME, $MESSAGE. noHost will use $TIMESTAMP, $MESSAGE. self will use $SYSLOG_pattern as defined by the user, by default $SYSLOG_pattern is defined same as bsd
If defined, derive more time information from the timestamp. This will convert time strings to epoch second using localtime function. This will slow down processing. This will be enabled if needed, such as when -min_date, -max_date, -report are used.
If enabled, will parse TAGs from Syslog Message. TAG syntax varys greatly from different devices/vendors. This module tries to parse out the common style but realizes it can not assume a common syntax so the user is allowed to define their own. See the Data Access section on TAGs for more information. Common style is considered to be: foo[123]:
($parse, $error) = $syslog->parse_syslog_line(<line>);
Method to parse the syslog line. If syslog line <line> is not given as argument then $_ is parsed.
The pattern used to parse the given line is a pattern made by the strings defined by $TIMESTAMP,$HOSTNAME,$MESSAGE. If -format is bsd, then $TIMESTAMP,$HOSTNAME,$MESSAGE are used to make the pattern. If -format is noHost, then $TIMESTAMP, $MESSAGE are used to make the pattern. These strings are used to make $SYSLOG_pattern
Some syslog daemons may prepend other information when writing syslog message to syslog file. parse_syslog_line will try to detect this by applying a regexp match for an RFC 3164 syslog message to $SYSLOG_pattern. The match will be treated as the syslog message, any string found before the match will be considered a preamble. The preamble will be parsed for receive time, syslog priority (facility.severity) and source IP address. This info is extracted and made avaliable to the user.
parse_syslog_line calls parse_syslog_msg to parse respective information. The string given to parse_syslog_msg is the string matched by $SYSLOG_pattern. Any facility or severity parsed is normalized to the strings listed in @FACILITY and @SEVERITY.
Syslog messages are the strings matched by $SYSLOG_pattern. Changing this string to something else allows the user to modify the parser.
The information parsed from the line given to parse_syslog_line is then checked against any filters.
If the filtering determines we keep the line the function checks if the -dump or -report options are enabled and performs the required task.
See Data Access Section for hash structure made by parse_syslog_line. Each call to this function clears the previous information stored.
The user has external control over the parser through the global variables $TIMESTAMP,$HOSTNAME,$MESSAGE, $SYSLOG_pattern. See the Data Access Section for requirements of these strings.
In list context a reference to a hash and error are returned. In scalar context, a reference to a hash is returned.
Events to Return Error:
@fields = parse_syslog_msg($msg, [$moreTime, $parseTag, <0|1>]); $fields[0] = timestamp portion of syslog message $fields[1] = hostname portion of syslog message $fields[2] = message portion of syslog message $fields[3] = error string
This function can be used externally as a function call. When called internally by parse_syslog_line the 4th argument is set to 1 and the function populates the respective data structure. If called externally, then will return in list context (timestamp, device, message, error) or in scalar context a reference to a hash whose keys are 'timestamp', 'device', 'message'. If an error occurs then everthing is undef and the error is a string detailing the error.
The given line is pattern matched against $SYSLOG_pattern and the information is gather from the $1, $2, $3, ... placeholders. See Data Access for syntax if defining your own $TIMESTAMP,$HOSTNAME,$MESSAGE, $SYSLOG_pattern strings.
If the -moreTime argument is true (non zero) then the $TIMESTAMP is picked apart for more information. This would be the -moreTime argument to parse_syslog_line when doing OOP, otherwise user must define on a user call.
If the -parseTag argument is true (non zero) then the parse_tag function attempts to parse a TAG. This would be the -parseTag argument to parse_syslog_line when doing OOP, otherwise user must define on a user call.
See Data Access section, Syslog Line Hash Reference for the syntax of the hash populated with the information parsed with parse_syslog_msg.
($tag, $pid, $content) = parse_tag(<string>);
This function will parse TAG and PID from the given string and return the TAG and PID and CONTENT. If the user defines $TAG_1 or $TAG_2 or $TAG_3, then those patterns are used to match. See Data Access for syntax of $TAG_x. Otherwise will look at beginning of the string for the common practice of some text string and pid, such as foo[1234]: content. The given string is assumed to be the MESSAGE portion of the syslog message, the MESSAGE portion can be made of TAG and CONTENT fields. The MESSAGE portion follows the HEADER which consist of TIMESTAMP and HOSTNAME, hostname field mandatory.
When used by the object, hash key/value entries are populated for the syslog line, see Data Access section. When used as a standalone function then TAG, PID and CONTENT fields are returned in list context.
This function is not external but is described since debugging will indicate this function.
($epoch, $date, $facility, $severity, $srcIP) = parse_preamble($preamble);
Some syslog daemon will prepend information to the RFC3164 format. This information is:
When used by the object, hash key/value entries are populated for the syslog line, see Data Access section.
This function will attempt to parse this information out and return in list context, the receive time in epoch seconds, the local receive time, the syslog message facility, the syslog message severity, the source IP address. Date information is assumed to be delimited with dash '-', time information is delimited with ':'. Since the syntax of this information is unique, any one of the fields are parsed for, thus they can be in any order in the preamble.
Constructor to create object that define the fields used in syslog messages to be sent from the localhost. Arguments define all portions of a RFC 3164 Syslog message. Message is sent when &send_message is called.
Any argument can be defined now or when calling &send_message. This allows the user to set values that are static for their needs or change dynamically each time a message is sent.
($syslog, $error) = Syslog->send( -server => <server IP>, [-port => <destination port>,] [-proto => <udp|tcp>,] [-facility => <facility string>,] [-severity => <severity string>,] [-timestamp => <message timestamp>,] [-device => <device name>,] [-tag => <tag>,] [-pid => <tag PID>,] [-content => <syslog messsage content>,] [-message => <syslog message>,] [-strict => <0|1>,] [-noHost => <0|1>,] [-hostname => <0|1>,] [-noTag => <0|1>,] [-debug => <0-5>,] );
Destination Syslog Server IP Address. Default 127.0.0.1
Destination Port. Default is 514. On UNIX systems, ports 0->1023 require user to be root (UID=0).
IP protocol to use, default is udp.
Syslog FACILITY (text) to use. Default is 'user'.
Syslog SEVERITY (text) to use. Default is 'debug'.
TIMESTAMP to put in to syslog message. Default is current time. Syntax is Mmm dd hh:mm:ss or Mmm d hh:mm:ss
Tell function to insert HOSTNAME field into syslog message. If 1, HOSTNAME field is not inserted into syslog message. If 0, HOSTNAME field is determined by -device or -hostname arguments. Default is 0.
HOSTNAME to put in to syslog message. If not given then -hostname determines HOSTNAME, if -hostname is not true, then 'netdevsyslog' is used for HOSTNAME field.
Use sytem hostname value for HOSTNAME field, if -device does not give a hostname to use, then a call to the systems hostname function is called to get value for HOSTNAME field of syslog message. If -device and -hostname are not used then 'netdevsyslog' is used for HOSTNAME field.
Tell function to insert a TAG into the syslog message. If 1, do not put a TAG into syslog message. If 0, insert a TAG into syslog message.
Syslog message TAG to insert into syslog message. If this is given and -noTag = 0 then this string will be used as the TAG of the syslog MESSAGE, a single space will separate it from the CONTENT. This value will not be used if -message is given. If not given and -noTag = 0 the function will create a TAG with the syntax of 'NetDevSyslog[$pid]:', If -pid is given. Otherwise 'NetDevSyslogp:[$$]', where system PID is used.
Syslog message TAG PID to use. If given, will create NetDevSyslog[$pid], otherwise the system PID for the the current script will be used NetDevSyslogp[$$]. This value will not be used if -message is given.
String to use for syslog message CONTENT. The message portion of a syslog message is defined as a TAG and CONTENT, with TAG not being mandatory. This field will be combined with -tag and -pid to create the syslog MESSAGE portion.
String to use as syslog MESSAGE portion. User may opt to not use the -tag, -pid, -content message and just pass in the complete MESSAGE portion to follow the TIMESTAMP and HOSTNAME fields. If this is given it will have precedence over -content.
By default strict syntax is enforced, this can be disabled with -strict 0. Strict rules allow message to be no longer than 1024 characters and TAG within the message to be no longer than 32 characters.
Function will create a RFC 3164 syslog message and send to destination IP:port. For values not defined by user, defaults will be used. The same arguments given for the constructor 'send' apply to this function. Thus any value can be changed before transmission.
($ok, $error) = $syslog->send_message( [-server => <server IP>], [-port => <destination port>,] [-proto => <udp|tcp>,] [-facility => <facility string>,] [-severity => <severity string>,] [-timestamp => <message timestamp>,] [-device => <device name>,] [-tag => <tag>,] [-pid => <tag PID>,] [-content => <syslog messsage content>], [-message => <syslog message>,] [-strict => <0|1>], [-noHost => <0|1>], [-hostname => <0|1>], [-noTag => <0|1>], [-debug => <0-5>], );
See above send method for descriptions of send_message options.
For any error detected, the message will not be sent and undef returned. For each message sent, the socket is opened and closed.
In list context the status and error are returned, in scalar context just the status is returned. If the message is sent successfully, then status is 1, otherwise undef. If an error occurs then the error variable is a descriptive string, otherwise undef.
Using the options/arguments of send and send_message the syslog message is created with the following logic:
PRI (decimal) is calculated from FACILITY and SEVERITY values, defaults are used if not given in function call.
TIMESTAMP is either given with -timestamp or taken from local system time.
HOSTNAME is created if -noHost set to 0. If -device given, then this becomes HOSTNAME. Else if -hostname is 1, then system hostname is used. Else 'netdevsyslog' is used. If -noHost is 1 then no HOSTNAME field is put into the Syslog message.
MESSAGE is created either from user giving the full message with the -message argument. Otherwise MESSAGE is created by combining TAG, PID, CONTENT. CONTENT is the message of the sylog line.
CONTENT creation follows this:
if -content given, then CONTENT is the given string if -noTag =0 if -tag and -pid are defined TAG becomes tag[pid]: if -tag only then TAG is the argument string if -pid only then NetDevSyslog[pid]: otherwise NetDevSyslogp[$$] otherwise CONTENT becomes the default message.
SYSLOG MESSAGE to be put on the wire is then created by joining the different variables, $TIMESTAMP, [$HOSTNAME], [$TAG], $MESSAGE | $CONTENT
Constructor to create object that listens on desired port and prints out messages received. Message are assumed to be syslog messages. Messages can also be forward unaltered to a defined address:port.
($syslog, $error) = Syslog->listen( [-port => <port>,] [-proto => <udp|tcp>,] [-maxlength => <max message length>,] [-packets => <integer>,] [-verbose => <0|1|2>,] [-report => <0|1>], [-fwd_server => <ip>], [-fwd_port => port>,] [-fwd_proto => <udp|tcp>,] );
If -report or -verbose <true> is used, then a parse oject is created and the same options that can be given to the parse object can be given to this object.
Message received will be printed to STDOUT.
On UNIX systems, if -port is less than 1024 then the the user must have UID=0 (root). To listen on ports 1024 and above any userid is allowed.
CTRL-C ($SIG{INT}) is redefined to shutdown the socket and then return control back to caller, it will not exit your program.
If -report option is enabled, then a reference to object that can be used to access %STATS will be returned.
Otherwise a counter value indicating the number of messages received is returned.
Local port to listen for messages. Messages are assumed to be syslog messages. Some OS's may require root access. Default is 514.
Protocol to use. Default is udp.
Max message length. Default is 1024
Shutdown the socket listening on after N packets are received on the given port. At least one packet must be received for packet count to be checked.
Verbosity level 0-3
Perform same reporting as the parse method does. All arguments to the parse method can be used on this method. Unlike the parse method, reporting is off by default for listen method.
Forward received message to address given. Message is sent as it is received off the wire. Dotted decimal IP address or DNS name can be given.
Define TCP port for forward message to be sent. Default is 514
Define transport protocol for forwarded message to be sent. Default is UDP. String 'udp' or 'tcp' can be given.
Initialize the hash storing the current syslog line information.
$syslog->init();
Function to loop through all filehandles opened for dumping a syslog line to a device specific file. The parsing function will take care of closing any files created with the -dump option, this is available to give the user the control if needed.
$syslog->close_dumps();
Function to convert epoch seconds, in the hash created with -report option, to a date string of Mmm/dd/yyyy hh:mm:ss. This function acts on the hash and convert epoch min, max time value for the whole syslog file and per each device. If you reference the hash before running this function you will not get the date string. This process is kept separate to save time in waiting for this to complete if done during parsing, since we only need to do it after the min and max are found. The syntax is purposely different than the syntax of a sylog message, but does contain the same information with a year value added.
&syslog_stats_epoch2datestr;
Function to convert epoch seconds to a RFC 3164 syslog message timestamp. If epoch seconds not given, then current time is used.
$timestamp = epoch_to_syslog_timestamp($epoch);
Function to convert epoch seconds to a common date string. If epoch seconds not given, then current time is used.
$date_str = epoch_to_datestr($epoch)
Date string format Mmm/dd/yyyy hh:mm:ss
Function to convert date given for a filter to epoch seconds.
$epoch = date_filter_to_epoch(<mm/dd/yyyy [hh:mm:ss]>);
Function to validate that a given timestamp matches the syntax defined by RFC 3164. If valid, then '1' is returned, if invalid then '0' is returned.
$ok = validate_timestamp_syntax($timestamp);
Function to create @TIMESLOTS given the min/max epoch seconds and the interval. Will start at min epoch value and increment until reaching or exceeding the max epoch value. For each increment an index is made based on the min epoch for that interval. The index is created with &epoch_to_datestr.
make_timeslots($min_epoch, $max_epoch, $interval);
Min and max values are mandatory and are checked to be greater or less than the other value. If $interval is not given, function defaults to 60 seconds.
The created list is built as such
@TIMESLOTS = ([$index, min_epoch, $max_epoch], ...);
This list can be used to group syslog messages to a specific timeslot. From the syslog line we have epoch seconds, this list provides a range to check the epoch seconds against and the index for that range.
Function that takes a given epoch second value and returns the timeslot index value for that value from @TIMESLOTS.
$index = epoch_timeslot_index($epoch);
If no match is found, undef is returned.
Function to take a character string representing a facility and return a normalize string contained in @FACILITY.
$facility = normalize_facility($facility);
If given string is not normailized, it is returned
Function to take a character string representing a severity and return a normalize string contained in @SEVERITY.
$severity = normalize_severity($severity);
Function to decode PRI in decimal format to a Facility and Severity. Can accept either decimal number or decimal number bounded by '<' '>'.
In list context will return list of information, in scalar context will return respective Facility and Severity strings joined with '.'.
@pri = decode_PRI($pri_dec); $PRI = decode_PRI($pri_dec); $pri[0] PRI decimal value $pri[1] Facility decimal value $pri[2] Severity decimal value $pri[3] PRI character string (join facility and severity string) $pri[4] Facility charater string $pri[5] Severity charater string
Given PRI value is checked to be between 0 and 191. If not, then undef is returned in scalar context and for list values any decimal number is -1, P?, F?, S? for PRI, Facility Severity character strings respectively
Set the value used by methods and functions of this module to the current year as known by localtime. Syslog message timestamps do not conatain year information. A user may need to change this when looking at a syslog from a different year.
If no value is given, then the current year is assumed, otherwise the year is set to the argument.
$syslog->set_year(2003); # set year to 2003 $syslog->set_year(); # set year to ((localtime)[5]) + 1900
Return reference to %STATS, this is the hash created with the -report option to the parser and listener.
Return reference to @DEVICES, list of HOSTNAMEs parsed. This is created with the -report option to the parser and listener.
Return reference to @FACILITY, list of FACILITIES parsed. This is created with the -report option to the parser and listener.
Return reference to @SEVERITY, list of SEVERITIES parsed. This is created with the -report option to the parser and listener.
Return reference to @TAGS, list of TAGS parsed This is created with the -report option to the parser and listener.
Return reference to @TIMESLOTS, list of time slots made by make_timeslot function.
Return last error.
Return error counter value, incremented each time the parser errors.
Return filter counter value, incremented each time a line is filtered
Return parser count, increments each time a line is given to parser
List of all syslog facilities strings as defined by RFC 3164. Any facility string parse or given by the user is normalized to strings found in this list.
List of all syslog severities strings as defined by RFC 3164. Any severity string parse or given by the user is normalized to strings found in this list.
Hash whose keys are syslog facility strings and whose value is the decimal representation of that facility.
Hash whose keys are syslog severity strings and whose value is the decimal representation of that severity.
The pattern used to parse TIMESTAMP from RFC 3164 syslog message.
(([JFMASONDjfmasond]\w\w) {1,2}(\d+) (\d{2}:\d{2}:\d{2}))
$1 = TIMESTAMP
$2 = Month string
$3 = Month day (decimal)
$4 = hh:mm::ss
The patterm used to parse HOSTNAME from RFC 3164 syslog message.
([a-zA-Z0-9_\.\-]+)
$1 = HOSTNAME
The user defined pattern used to parse a TAG from RFC 3164 syslog message. If any of these are defined, then the TAG is to be parsed against these patterns, otherwise the modules regexp pattern will be used.
$TAG_1 $1 = task, $2 = pid, $3 = content
$TAG_2 $1 = task, $2 = content, no pid
$TAG_3 $1 = task, $2 = pid, $3 = content
The pattern used to parse MESSAGE from RFC 3164 syslog message.
(.+)$
$1 = MESSAGE
The pattern used to parse any RFC 3164 syslog message. Combination of $TIMESTAMP, $HOSTNAME, $MESSAGE are used to parse the different parts from RFC 3164 syslog message. The user can define this by defining the individual variables that make this up or just define this directly.
If used when -format is 'self' then
$5 = HOSTNAME
$6 = MESSAGE
The hash reference returned by function parse_syslog_line has the following keys:
($hash_ref, $error) = $syslog->parse_syslog_line($message); $hash_ref->{'line'} current line from syslog file {'timestamp'} timestamp from syslog message {'device'} device name from syslog message {'message'} syslog message, from after devname {'month_str'} month from syslog message timestamp (Jan,Feb,...) {'month'} month index 0->11 {'day'} day from syslog message timestamp {'time_str'} hh:mm:ss from syslog message timestamp {'hour'} hh from syslog message timestamp {'min'} mm from syslog message timestamp {'sec'} ss from syslog message timestamp {'year'} year assumed from localtime {'epoch'} epoch time converted from syslog message timestamp {'wday'} wday integer derived from epoch (0-6) = (Sun-Sat) {'wday_str'} wday string converted, (Sun, Mon, ...) {'date_str'} syslog message {'epoch'} convert to common format {'tag'} syslog message content tag {'pid'} syslog message content tag pid {'content'} syslog message content after tag parsed out {'preamble'} string prepended to syslog message {'rx_epoch'} extra info: rx time epoch {'rx_timestamp'} extra info: rx timestamp {'rx_priority'} extra info: priority (text) {'rx_facility'} extra info: syslog facility (text) {'rx_severity'} extra info: syslog severity (text) {'srcIP'} extra info: src IP address {'rx_epoch'} extra info: rx time epoch {'rx_date_str'} extra info: rx time date string {'rx_time_str'} extra info: rx time (hh:mm:ss) {'rx_year'} extra info: rx time year value {'rx_month'} extra info: rx time month value {'rx_month_str'} extra info: rx time month value string (Jan,Feb,..) {'rx_day'} extra info: rx time day value {'rx_wday'} extra info: rx time weekday (0-6) (Sun, Mon,..) {'rx_hour'} extra info: rx time hour value {'rx_min'} extra info: rx time minute value {'rx_sec'} extra info: rx time second value
key = timestamp is $TIMESTAMP
key = device is $HOSTNAME
key = message is $MESSAGE
key = month, day, time_str, hour, minute, sec is parsed from $TIMESTAMP. month_str is derived
key = tag and pid may come from $TAG_x if defined
key = content is the message part after a TAG, if parsed, otherwise the whole message after $HOSTNAME for bsd format, $TIMESTAMP for noHost format.
key = preamble is the entire preamble string if found.
key = rx_* is any information found and parsed from the preamble
Multi-level hash (HoH) that store statisticis. This hash is created as each line is parsed if -report is enabled. This only represent some basic stats that I thought everyone would want. A user can derive their own by examining the different fields in hash reference returned by parse_syslog_line.
All of the values listed below are incremented (counter). Strings enclosed in '<' '>' denote keys derived from information found in the syslog file, in other words they are variable whereas the single quoted strings are constant (hardcoded). $STATS{'syslog'}{'messages'} {'min_epoch'} {'max_epoch'} {'min_date_str'} {'max_date_str'} {'tag'}{<$tag>}{'messages'} {'facility'}{<$rx_facility>}{'messages'} {'severity'}{<$rx_severity>}{'messages'} $STATS{'device'}{<$dev>}{'messages'} {'min_epoch'} {'max_epoch'} {'min_date_str'} {'max_date_str'} {'tag'}{<$tag>}{'messages'} {'facility'}{<$rx_facility>}{'messages'} {'severity'}{<$rx_severity>}{'messages'}
@TIMESLOTS is a list (AoA) of time intervals ranging from the min to max value provided to &make_timeslots function. A @TIMESLOTS element contains 3 values
@TIMESLOTS = ([index, min_epoch, max_epoch], ...); index - Unique string created to indicate start of timeslot Mmm/dd/yyyy hh:mm min_epoch - is begining of the timeslot interval in epoch seconds. max_epoch - is ending of the timeslot interval in epoch seconds.
List of devices found. Created when -report is true. When a device is firsted learned, its device name as known from the syslog message is pushed on to this list.
List of tags found. Created when -report is true. When a tag is firsted learned, its name as known from the sylog message is pushed on to this list.
Counter for each error occuring in parser.
Counter for each line filtered by parser.
Counter for each line parsed by parser.
RFC 3164 describes the syntax for syslog message. This modules intends to adhere to this RFC as well as account for common practices.
As described in the RFC, 'device' is a machine that can generate a message. A 'server' is a machine that receives the message and does not relay it to any other machine. Syslog uses UDP for its transport and port 514 (server side) has been assigned to syslog. It is suggested that the device source port also be 514, since this is not mandatory, this module does not enforce it.
Section 4.1 of RFC 3164 defines syslog message parts, familiarity with these descriptions will give the user a better understanding of the functions and arguments of this module. Maximum length of a syslog message must be 1024 bytes. There is no minimum length for a syslog message. A message of 0 bytes should not be transmitted.
4.1.1 PRI Part of RFC 3164 describes PRI. The PRI represents the syslog Priority value which represents the Facility and Severity as a decimal number bounded by angle brackets '<' '>'. The PRI will have 3,4 or 5 characters. Since two characters are always the brackets, the decimal number is then 1-3 characters.
The Facility and Severity of a message are numerically coded with decimal values.
Numerical Facility Code 0 kernel messages 1 user-level messages 2 mail system 3 system daemons 4 security/authorization messages (note 1) 5 messages generated internally by syslogd 6 line printer subsystem 7 network news subsystem 8 UUCP subsystem 9 clock daemon (note 2) 10 security/authorization messages (note 1) 11 FTP daemon 12 NTP subsystem 13 log audit (note 1) 14 log alert (note 1) 15 clock daemon (note 2) 16 local use 0 (local0) 17 local use 1 (local1) 18 local use 2 (local2) 19 local use 3 (local3) 20 local use 4 (local4) 21 local use 5 (local5) 22 local use 6 (local6) 23 local use 7 (local7) Note 1 - Various operating systems have been found to utilize Facilities 4, 10, 13 and 14 for security/authorization, audit, and alert messages which seem to be similar. Note 2 - Various operating systems have been found to utilize both Facilities 9 and 15 for clock (cron/at) messages. Numerical Severity Code 0 Emergency: system is unusable 1 Alert: action must be taken immediately 2 Critical: critical conditions 3 Error: error conditions 4 Warning: warning conditions 5 Notice: normal but significant condition 6 Informational: informational messages 7 Debug: debug-level messages
Priority is calculated as: (Facility*8) + Severity. After calculating the Priority, bound it with barckets and its now a PRI. For example a daemon debug would be (3*8)+7 => 31 Priority, PRI <31>.
The header portion contains a timestamp and the device name or IP. The device name is not mandatory.
The TIMESTAMP immediately follows the trailing ">" from the PRI when received on the wire. The TIMESTAMP is separated from the HOSTNAME by single space characters.
The TIMESTAMP field is the local time of the sending device and is in the i format of 'Mmm dd hh:mm:ss'.
Mmm is the month abbreviation, such as: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec. dd is day of month. If numeric day value is a single digit, then the first character is a space. This would make the format 'Mmm d hh:mm:ss'. hh:mm::ss are hour minute seconds, 0 padded. Hours range from 0-23 and minutes and seconds range from 0-59.
A single space charater must follow the the TIMESTAMP field.
The HOSTNAME is separated from the precedding TIMESTAMP by single space character. The HOSTNAME will be the name of the device as it knows itself. If it does not have a hostname, then its IP address is used.
The MSG part will fill the rest of the syslog packet. The MSG part is made of two parts the TAG and CONTENT. The TAG value is the name of the originating process and must not exceed 32 characters.
Since there is no specific rule governing TAGs, the user of this module is allowed to define 3 TAG patterns to be matched. Otherwise the module trys to extract common practices.
The CONTENT is the details of the message.
Examples are also in ./examples directory of distribution. This directory also contains syslog generator script that is a bit more elaborate than a simple example but is good for generating syslog message from a range of dummy host name. Can generate a file or be used to send to a server. The comments in the code should give you a general idea of what is being done and what the varaibles mean.
#!/usr/bin/perl # Perl script to test Net::Dev::Tools::Syslog listen use strict; use Net::Dev::Tools::Syslog; my ($listen_obj, $error, $ok); my $port = 7971; my $proto = 'udp'; # create object to listen # CTRL-C will close sock and return to caller ($listen_obj, $error) = Syslog->listen( -port => $port, -proto => $proto, #-verbose => 3, #-packets => 150, #-parseTag => 1, ); unless ($listen_obj) { printf("ERROR: syslog listen failed: %s\n", $error); exit(1); } exit(0);
#!/usr/bin/perl # Perl script to test Net::Dev::Tools::Syslog sending # use strict; use Net::Dev::Tools::Syslog; my ($send_obj, $error, $facility, $severity, $ok, ); my $server = '192.168.1.1'; my $port = 7971; my $proto = 'udp'; my $test_send_all = 1; my $sleep = 0; my $pid = $$; # create send object ($send_obj, $error) = Syslog->send( -server => $server, -port => $port, -proto => $proto, ); unless ($send_obj) { myprintf("ERROR: Syslog send failed: %s\n", $error); exit(1); } # send syslog message printf("Sending syslog to %s:%s proto: %s pid: %s\n", $server, $port, $proto, $pid ); # send all syslog type message if ($test_send_all) { foreach $facility (@Syslog::FACILITY) { foreach $severity (@Syslog::SEVERITY) { #printf("send message: %-10s %s\n", $facility, $severity); ($ok, $error) = $send_obj->send_message( -facility => $facility, -severity => $severity, -hostname => 1, -device => 'myTestHost', -noTag => 0, #-tag => 'myTag', -pid => 1, -content => 'my syslog message content', ); if(!$ok) { printf("ERROR: syslog->send_msg: %s\n", $error); } sleep $sleep; } } } else { ($ok, $error) = $send_obj->send_message( -hostname => 1, ); if(!$ok) { printf("ERROR: syslog->send_msg: %s\n", $error); } } exit(0);
#!/usr/bin/perl # Perl script to test Net::Dev::Tools::Syslog parsing # use strict; use Net::Dev::Tools::Syslog; # get sylog file from cli my $syslog_file = shift || die "usage: $0 <syslog file>\n"; my ($syslog_obj, $error, $parse_href, $report_href, $fh, $device, $tag, $facility, $severity, ); # create syslog parsing object ($syslog_obj, $error) = Syslog->parse( -report => 1, -parseTag => 1, -dump => './dump2', -debug => 0, -moreTime => 1, -format => 'noHost', ); unless ($syslog_obj) { printf("sylog object constructor failed: %s\n", $error); exit(1); } # open syslog file to parse printf("parse syslog file: %s\n", $syslog_file); open ($fh, "$syslog_file") || die "ERROR: open failed: $!\n"; while(<$fh>) { ($parse_href, $error) = $syslog_obj->parse_syslog_line($_); unless ($parse_href) { printf("ERROR: line %s: %s\n", $., $error); } } close($fh); printf("parse syslog file done: %s lines\n", $.); # convert epoch time in report hash &syslog_stats_epoch2datestr; # reference report hash and display $report_href = &syslog_stats_href; # stats for entire syslog file printf("Syslog: messages %s %s -> %s\n\n\n", $report_href->{'syslog'}{'messages'}, $report_href->{'syslog'}{'min_date_str'}, $report_href->{'syslog'}{'max_date_str'}, ); # stats for each device found in syslog foreach $device (keys %{$report_href->{'device'}}) { printf("Device: %s messages: %s %s -> %s\n", $device, $report_href->{'device'}{$device}{'messages'}, $report_href->{'device'}{$device}{'min_date_str'}, $report_href->{'device'}{$device}{'max_date_str'}, ); printf(" Tags:\n",); foreach $tag (keys %{$report_href->{'device'}{$device}{'tag'}}) { printf(" %8s %s\n", $report_href->{'device'}{$device}{'tag'}{$tag}{'messages'}, $tag ); } printf(" Facility:\n",); foreach $facility (keys %{$report_href->{'device'}{$device}{'facility'}}) { printf(" %8s %s\n", $report_href->{'device'}{$device}{'facility'}{$facility}{'messages'}, $facility ); } printf(" Severity:\n",); foreach $severity (keys %{$report_href->{'device'}{$device}{'severity'}}) { printf(" %8s %s\n", $report_href->{'device'}{$device}{'severity'}{$severity}{'messages'}, $severity ); } printf("\n"); } exit(0);
sparsons@cpan.org
Copyright (c) 2004-2006 Scott Parsons All rights reserved. This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself.
To install Net::Dev::Tools::Syslog, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Net::Dev::Tools::Syslog
CPAN shell
perl -MCPAN -e shell install Net::Dev::Tools::Syslog
For more information on module installation, please visit the detailed CPAN module installation guide.