Jon Schutz > File-Tail-Dir > File::Tail::Dir

Download:
File-Tail-Dir-0.16.tar.gz

Dependencies

Annotate this POD

CPAN RT

New  2
Open  0
View/Report Bugs
Module Version: 0.16   Source  

NAME ^

File::Tail::Dir - Tail all matching files in a given set of directories

SYNOPSIS ^

Typical usage:

  use File::Tail::Dir;

  my $tailer = File::Tail::Dir->new(  filter => qr/.*access.*log$/,
                                        processor => sub { 
                                              my ($filename, $lines) = @_; 
                                              ... 
                                              },
                                     );
  $tailer->watch_files();

Or, subclass:

  package My::Tailer;
  use Moose;
  extends 'File::Tail::Dir';

  # override standard 'process' method
  sub process {
    my ($self, $filename, $lines) = @_;

      ...
    }

DESCRIPTION ^

Monitors and processes any lines appended to the end of one or more files in a given list of directories, keeping state between sessions, and using kernel notification of file change events to maximise efficiency.

A list of directories is given to monitor, and filtering/exclude expressions allow just the files of interest to be selected for notification when lines are appended.

This module was originally created to support lossless logging of many Apache web server log files to a Scribe-based logging subsystem (see File::Tail::Scribe, Log::Dispatch::Scribe), and hence key requirements are to keep state, to be able to resume from the last known state between interruptions (like server reboots), and to follow the renaming and creation of new files during log rotation.

ALTERNATIVES ^

Like File::Tail::Multi, File::Tail::App, File::Tail::FAM, File::Tail, File::Tail::Dir is yet another module to tail and process one or more files. What File::Tail::Dir provides that is different is:

CONSTRUCTOR ^

new

  $tailer = File::Tail::Dir->new(%options);

Options are as follows:

File Selection Options
  • directories => \@paths

    Spefify the directory paths that should be watched for changes.

  • filter => qr/.../

    This is an optional regular expression that will be used to check if a file is of interest. This filter is only applied to files. By default, all files are included.

  • exclude => [...]

    An optional list of paths to exclude. This list can contain either plain strings or regular expressions. If you provide a string it should contain the full filesystem path to be excluded.

    The paths can be either directories or specific files. If the exclusion matches a directory, all of its files and subdirectories are ignored.

  • follow_symlinks => $bool

    By default, symbolic links in the filesystem are ignored. Set this to true to follow them.

    If symlinks are being followed, symlinks to files and directories will be followed. Directories will be watched, and changes for directories and files reported.

    (Note that if this is set to true, the presence of circular link references may cause problems due to a behaviour in the underlying File::ChangeNotify).

Processing Options
  • processor => sub { my ($filename, $lines) = @_; ... }

    Specifies the callback function to process the arrayref of lines appended to the given file.

    The default action is to print the lines to standard output.

Miscellaneous Options
  • sleep_interval => $number

    Where kernel file change event notification is not available, this argument controls the interval between checks for file changes. The value is a number in seconds. The default is 2 seconds.

  • statefilename => $filename

    Sets the name of the file used to store state between sessions. Defaults to '.filetaildirstate' in the working directory.

  • autostate => 1|0

    Set to 1 to automatically store the state file periodically. Defaults to 0.

  • autostate_delay => $number

    If autostate is enabled, sets the minimum delay in seconds between when the state file is automatically saved. If set to 0, state is saved on every file change.

  • no_init => 1|0

    If no_init is set, the initial contents of the monitored files is skipped and only the changed contents (since creating the new File::Tail::Dir instance) is sent.

    The default behaviour (no_init = 0) is for lossless logging, i.e. to send any contents that has been appended to existing files since last run, and to see all files without existing state as new files and therefore to send the whole contents of those files before monitoring for new changes.

  • max_age => $age_in_seconds

    Maximum time in seconds to keep a filehandle open if no changes on the filehandle have been seen. Defaults to 3600 sec.

  • max_lines => $number

    Maximum number of lines to process from a given file at a time. This sets a limit on the maximum amount of memory to use, particularly in the case of processing large files for the first time. Defaults to 10000.

  • watchdog_signal_name

    Name of signal to use for watchdog. Defaults to USR1 on most systems, INT on Windows.

METHODS ^

watch_files

  $tailer->watch_files()

Blocking call to watch for changes in files forever. When a change is detected, the process() method is called.

process

This is the method invoked by watch_files() on file change, which in turn invokes the callback function specified by the 'processor' option. If no callback has been provided, it will print the filename and changed contents to standard output by default. This method may be overridden in a subclass instead of using the processor option. Its signature is:

  sub process {
     my ($self, $filename, $lines) = @_;
     ...
  }

The return value is ignored.

running

   $running = $tailer->running();  # get status
   $tailer->running(0); # exit event loop

This is set to true when the event loop in watch_files() is entered. It can be set to false (e.g. via a signal or in the process() method) to cause watch_files() to return after processing the current batch of events.

Note that if running is set to false via signal, the event loop will not be exited until a new file change event is seen.

save_state

  $tailer->save_state();

Saves internal state to file. This is always called when watch_files() completes normally, and is called periodically if autostate is enabled. However, you may wish to call it directly if you have installed a signal handler that might not allow watch_files() to complete.

__PACKAGE__->install_signal_handlers

    File::Tail::Dir->install_signal_handlers();

Installs default signal handlers. Once installed, on receiving any of HUP, INT, QUIT or TERM, all instances of File::Tail::Dir will attempt to save their state, and then exit.

If used, this should be called before any File::Tail::Dir instance is created.

AUTHOR ^

Jon Schutz, <jon at jschutz.net> http://notes.jschutz.net

BUGS ^

Please report any bugs or feature requests to bug-file-tail-dir at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Change-Dir. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT ^

You can find documentation for this module with the perldoc command.

    perldoc File::Tail::Dir

You can also look for information at:

COPYRIGHT & LICENSE ^

Copyright 2012 Jon Schutz, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

syntax highlighting: