Dezi::Bot::Handler::FileCacher - web crawler handler that caches files
use Dezi::Bot::Handler::FileCacher;
my $handler = Dezi::Bot::Handler::FileCacher->new(
dsn => "DBI:mysql:database=$database;host=$hostname;port=$port",
username => 'myuser',
password => 'mysecret',
root_dir => '/path/to/site/mirror',
);
$handler->handle( $swish_prog_doc );
The Dezi::Bot::Handler::FileCacher writes each doc to the filesystem, managing its progress and status via DBI.
Returns a new Dezi::Bot::Handler::FileCacher object. config must have:
Passed to DBI->connect.
Passed to DBI->connect.
Passed to DBI->connect.
The table_name value will be used to insert rows. Defaults to dezi_file_cache.
The quote value will be used to quote column names on insert. Defaults to false.
The quote_char value will be used when quote is true. Defaults to backtick.
Base path for writing cached files.
Internal method to initialize object.
Metadata from doc is stored via DBI and doc is written to disk.
Write the key/value pairs of record to the database represented by dsn. record is metadata about the cached file. Timestamps are automatically added to record by write_record().
Returns the id (primary key) of the new record.
Callable as a function or class method. Returns string suitable for initializing a dezi_filecache SQL table.
Example:
perl -e 'use Dezi::Bot::Handler::FileCacher; print Dezi::Bot::Handler::FileCacher::schema' |\ sqlite3 dezi.index/bot.db
Peter Karman, <karman at cpan.org>
Please report any bugs or feature requests to bug-dezi-bot at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Dezi-Bot. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
You can find documentation for this module with the perldoc command.
perldoc Dezi::Bot
You can also look for information at:
Copyright 2013 Peter Karman.
This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.
See http://dev.perl.org/licenses/ for more information.
