Tail::Stat - Real-time log statistics server
It's often necessary to collect some statistics data for following analysis or processing. Common case are various monitoring tools, like a MRTG, Nagios or Zabbix. Some services may be examined by special commands or protocols, other may not. But often, information we are interested in can be extracted from server logs. This software helps to extract, accumulate and provide this information to monitoring tool via simple, easy parseable protocol.
Tail::Stat has a plugin structure. Each plugin supports logs processing for a specific service. Main executable (called tstatd) works as a long running background process (daemon) with TCP listen socket for querying about collected statistics. There is no any configuration files, all required parameters tstatd takes from command line options. One running instance of tstatd can process many similar log files simultaneously. It agregates extracted parameters into zones. Zones are just namespaces for grouping this values. For collecting parameters from other kind of service you have to run other instance of tstatd.
Fundamentally all measured values can be separated into two principally different groups: counters and gauges.
processing web-server logs we want to calculate two parameters: total processed HTTP requests and average time elapsed per request.
The first goal can be achieved by simple incremented counter,
but the second is a little harder.
We have to summarize elapsed times and then divide this amount onto request count.
But we have to do this for a small time slot (usually comparable with our monitoring tool polling interval).
This kind of calculations is supported by Tail::Stat and calls sliding windows calculations.
Tail::Stat operate with a set of a small windows.
First window (called current) accumulates current data.
After a fixed period of time (
--window-size) a window is closing (special handler executing),
new window creating and setting as current and last of closed windows is removing.
Total number of windows can be adjusted by special option (
For example: our monitoring tool has a polling interval about 10 minutes (600 seconds).
We want to provide it average response time for last 600 seconds respectively.
Appropriate window size can be set as 10 seconds with keeping values for 60 windows (and this are default values).
Querying accumulated data is available via simple TCP-based protocol. Protocol is line oriented (like an HTTP or SMTP).
Prints list of known zones. Zones specified via command line options marked with 'a:' prefix (active). Zones restored from a database file, but not found in command options marked with 'i:' prefix (inactive).
Prints list of globs (wildcards) associated with zone. Applicable only to active zones.
Prints list of files currently processing for zone. Each file prefixed by current reading offset and size of file. Applicable only to active zones.
Wipes out an inactive zone or all inactive zones. Applicable only to inactive zones.
Prints out raw zone statistics.
Prints out formatted zone statistics.
Closes client connection.
Please report any bugs or feature requests to
bug-tail-stat at rt.cpan.org,
or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Tail-Stat.
I will be notified,
and then you'll automatically be notified of progress on your bug as I make changes.
You can find documentation for this module with the perldoc command.
You can also look for information at:
Oleg A. Mamontov,
This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.
See http://dev.perl.org/licenses/ for more information.