WWW::Bookmark::Crawler - Personal bookmark search engine
use WWW::Bookmark::Crawler;
$crawler = WWW::Bookmark::Crawler->new({
SOURCE => 'bookmarks.html',
DBNAME => 'mybookmark.db',
PEEK => 1,
TOKENIZER => \&my_tokenizer,
});
$crawler->peek();
$crawler->crawl();
$crawler->nopeek();
$crawler->query('Ars longa');
WWW::Bookmark::Crawler is a WWW spider and a search engine for personal bookmark. It first extracts links in either a browser-generated bookmark or a plain html file, then retrieves each page's content online and builds the index file. User can use this module to build a personal bookmark search engine.
Parameters:
User may feed it with either the name of bookmark file or reference to an array of urls.
The name of the index file.
This is passed on to LWP agent.
Ditto. Default is 10 seconds.
Set it to non-undef if user wants to see the debugging log dumping to STDOUT. Default is undef.
User may write an ad hoc tokenizer replacing the given one. WWW::Bookmark::Crawler uses OurNet::FuzzyIndex to play the role.
Starts fetching and building index file.
Returns an array of hashes of URLs and Titles related to the given terms. The default tokenizer treats space as intersection. This method builds an in-memory inverted file from index file when it appears the first time in a script.
No advanced IR skills are used.
Turns on the debugging output. Same effective as PEEK given to new.
Turns off the debugging information.
Sets the proxy server. Same effective as PROXY given to new.
Sets the TIMEOUT value. Same effective as PROXY given to new.
xern <xern@cpan.org>
Released under The Artistic License.
