Dezi::Bot::Handler::FileCacher - web crawler handler that caches files
use Dezi::Bot::Handler::FileCacher; my $handler = Dezi::Bot::Handler::FileCacher->new( dsn => "DBI:mysql:database=$database;host=$hostname;port=$port", username => 'myuser', password => 'mysecret', root_dir => '/path/to/site/mirror', ); $handler->handle( $swish_prog_doc );
The Dezi::Bot::Handler::FileCacher writes each doc to the filesystem, managing its progress and status via DBI.
Returns a new Dezi::Bot::Handler::FileCacher object. config must have:
Passed to DBI->connect.
The table_name value will be used to insert rows. Defaults to dezi_file_cache.
dezi_file_cache
The queue_name value will be inserted for each row in the database.
The quote value will be used to quote column names on insert. Defaults to false.
false
The quote_char value will be used when quote is true. Defaults to backtick.
Base path for writing cached files.
Internal method to initialize object.
Metadata from doc is stored via DBI and doc is written to disk.
Write the key/value pairs of record to the database represented by dsn. record is metadata about the cached file. Timestamps are automatically added to record by write_record().
Returns the id (primary key) of the new record.
Callable as a function or class method. Returns string suitable for initializing a dezi_filecache SQL table.
Example:
perl -e 'use Dezi::Bot::Handler::FileCacher; print Dezi::Bot::Handler::FileCacher::schema' |\ sqlite3 dezi.index/bot.db
Peter Karman, <karman at cpan.org>
<karman at cpan.org>
Please report any bugs or feature requests to bug-dezi-bot at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Dezi-Bot. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
bug-dezi-bot at rt.cpan.org
You can find documentation for this module with the perldoc command.
perldoc Dezi::Bot
You can also look for information at:
RT: CPAN's request tracker
http://rt.cpan.org/NoAuth/Bugs.html?Dist=Dezi-Bot
AnnoCPAN: Annotated CPAN documentation
http://annocpan.org/dist/Dezi-Bot
CPAN Ratings
http://cpanratings.perl.org/d/Dezi-Bot
Search CPAN
http://search.cpan.org/dist/Dezi-Bot/
Copyright 2013 Peter Karman.
This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.
See http://dev.perl.org/licenses/ for more information.
To install Dezi::Bot, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Dezi::Bot
CPAN shell
perl -MCPAN -e shell install Dezi::Bot
For more information on module installation, please visit the detailed CPAN module installation guide.