Rafael Porres Molina > SNMP-LogParser-1.15290 > logparser.pl

Download:
SNMP-LogParser-1.15290.tar.gz

Annotate this POD

View/Report Bugs
Source  

NAME ^

logparser.pl

SYNOPSIS ^

logparser.pl [-f configFile] [-p processes] [-s storeFile] [-o propertiesFile] [-l log4perlFile] [-k lockFile] [-I path/to/lib1 -I path/to/lib2]

Parses a log file

logparser.pl -h

Shows the help man page

logparser.pl -v

shows the version

DESCRIPTION ^

The logparser script is supposed to be used from cron and to parse log files every five minutes starting from the last position read. It should take in account files that are rotated.

The main configuration comes from the configuration file (see the -f switch in the OPTIONS section).

The main parsing of any log file should be accomplished by creating an inherited class from the class LogparserDriver which has methods for specifying the regular expression, the evalBegin, evalIterate and evalEnd method.

By default the process is the following:

SETTING UP THE LOGPARSER

# Create a subclass of the LogparserDriver.

You need to implement at least define the variable pattern (the regular expression), and the methods evalBegin, evalIterate (invoked for each line of the file) and evalEnd.

For an exact description of the methods please see LogparserDriver

# Create a configuration file for logparser.

See the -f option. But mainly you need to specify the log file to parse and the subclass of LogparserDriver to use.

# (Optional) Set up the log configuration in log4perl.conf

The default logging entry for logparser uses the tag "logparser" and the LogparserDriver uses "logparser.LogparserDriver" tag. That is any subclass of LogparserDriver (including LogparserDriver itself) uses as the logging tag: "logparser.classname". For more information about logging please see Log::Log4perl

# Set up the logparser to run from cron

This can usually be achieved by creating a cron entry like this (please check the syntax for your exact *nix system):

$ crontab -e

*/5 * * * * [ -x /usr/bin/logparser ] && /usr/bin/logparser

Please be aware that the logfile should be possible to read as the user you are running cron from.

PROCESS OF LOGPARSER

The logparser works as follows

OPTIONS ^

All the command line options override the options in the configuration file.

COMMAND LINE OPTIONS

-f|--file configuration file

Indicates the configuration file. There is no corresponding configuration file option. The default value is "/etc/logparser/logparser.conf".

-p|--process-number process number

Indicates how many concurrent processes should be run in parallel. The corresponding configuration file option is "processes".

The default value is 1.

This option is not implemented yet

-s|--store-file storeFile

Indicates in which file the %properties hash should be stored. This has will be stored in a Java properties file in pairs of key=value pairs

For more information please see the LogparserDriver page.

-l|--log4perl-file log4perlFile

Indicates the configuration file for the Log4Perl configuration file. The corresponding configuration file option is "log4perlFile". The default value is "/etc/logparser/log4perl.conf"

-k|lock-file lockFile

Indicates the lock file. The corresponding configuration file option is "lockFile". The default value is "/tmp/.logparser.lockfile"

-o|--properties-file

Indicates the properties file. The corresponding configuration file option is "propertiesFile". The default value is "/var/lib/logparser/logparser.properties"

-I|--include path/to/lib

Use this to include alternative paths to find Logparser Drivers to avoid setting shell variables when running logparser

-h|--help

Shows this help page

-v|--version

Shows the version of the script.

CONFIG FILE OPTIONS

The configuration tag used is "logparser::Default"

log4perl

This option specifies the log4perl settings for logs. See the Log::Log4perl documentation.

log

Specifies all the logs that should be parsed. Each "<KEY>" indicates a different log. The different entries that can be used are:

* log.<KEY>.name: name identifies the log entry. By default the name defaults to "<KEY>". Be aware that the name is used to identify the log position. That is if you change the name (or the key if you don't define the name) then the log will be parsed from the beginning

* log.<KEY>.file: This is the file that should be parsed. This file should always be defined.

* log.<KEY>.driver: This is the class that should be invoked to parse the file specified above. Please be aware that the class should be a subclass of LogparserDriver class

EXAMPLE ^

We will provide here a detailed example on how to parse a particular file:

Assume that we want to get the number of email messages sent and include the size of these email messages.

An example input line of the log file /var/log/maillog could be:

 Sep  4 11:50:03 localhost sendmail[4091]: k849o3DZ004091: from=root, size=236, class=0, nrcpts=1, msgid=<200609040950.k849o3DZ004091@localhost.localdomain>, relay=root@localhost

The output of the incremental parsing that we want to record should be registered in a file /var/lib/logparser/logparser.properties with the values:

 mailMessages=23
 sizeOfMailMessages=52354

The steps that we will follow are:

REQUIREMENTS and LIMITATIONS ^

INSTALLATION ^

Required Perl packages

The perl packages installed for this script are:

BUGS ^

TODO ^

- Allow for non line oriented parsing ($/)
- Save the configuration in the store hash and use only the parsing of options when specified in the command line.

SEE ALSO ^

strftime(3) man page for specifying the directory/log which should be parsed.
Log::Log4perl For the logging configuration
SNMP::LogparserDriver For the default driver for parsing logs

AUTHOR ^

Nito Martinez <Nito at Qindel dot ES>

5/5/2005

COPYRIGHT & LICENSE ^

Copyright 2007-2011 by Qindel Formacion y Servicios SL, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

syntax highlighting: