NAME

HTML::Index::Store - subclass'able module for storing inverted index files for the HTML::Index modules.

SYNOPSIS

    my $store = HTML::Index::Store->new( 
        MODE => 'r',
        COMPRESS => 1,
        DB => $db,
        STOP_WORD_FILE => $path_to_stop_word_file,
    );

DESCRIPTION

The HTML::Index::Store module is generic interface to provide storage for the inverted indexes used by the HTML::Index modules. The reference implementation uses in memory storage, so is not suitable for persistent applications (where the search / index functionality is seperated).

There are two subclasses of this module provided with this distribution; HTML::Index::Store::BerkeleyDB and HTML::Index::Store::DataDumper

CONSTRUCTOR OPTIONS

Constructor options allow the HTML::Index::Store to provide a token to identify the database that is being used (this might be a directory path of a Berkeley DB implementation, or a database descriptor for a DBI implementation). It also allows options to be set. Some of these options are then stored in an options table in the database, and are therefore "sticky" - so that the search interface can automatically use the same options setting used at creating time.

DB

Database identifier. Available to subclassed modules using the DB method call. Not sticky.

MODE

Either 'r' or 'rw' depending on whether the HTML::Index::Store module is created in read only or read/write mode. Not sticky.

STOP_WORD_FILE

The path to a stopword file. If set, the same stopword file is available for both creation and searching of the index (i.e. sticky).

COMPRESS

If true, use Compress::Zlib compression on the inverted index file. The same compression is used for searching and indexing (i.e. sticky).

STEM

An option, if set, causes the indexer to use the Lingua::Stem module to stem words before they are indexed, and the searcher to use the same stemming on the search terms (i.e. sticky). Takes a locale as an argument.

SOUNDEX

An option, if set, causes the searcher to use the Text::Soundex to expand a query term on search if an exact match isn't found. To work, this option needs to be set at indexing, so that entries for soundex terms can be added to the index (i.e. sticky). If this has been done, then a SOUNDEX option can be passed to the search function to ennable soundex matching for a particular query.

VERBOSE

An option which causes the indexer / searcher to print out some debugging information to STDERR.

NOPACK

An option which prevents the storer from packing data into binary format. Mainly used for debugging (sticky).

PUBLIC INTERFACE

These methods are used as an interface to the underlying store. Subclasses of HTML::Index::Store should implement "SUB-CLASSABLE METHODS", but can optionally directly subclass methods in the public interface as well.

index_document( $document )

Takes an HTML::Index::Document object as an argument, and adds it to the index.

deindex_document( $document )

Takes an HTML::Index::Document object as an argument, and removes it from the index.

search( $q )

Takes a search query, $q, and returns a list of HTML::Index::Document objects corresponding to the documents that match that query.

filter( @w )

Takes a list of words, and returns a filtered list after filtering (lowercasing, non-alphanumerics removed, short (<2 letter) words removed, stopwords, stemming).

SUB-CLASSABLE METHODS

init

Initialisation method called by the constructor, which gets passed the options hash (see "CONSTRUCTOR OPTIONS"). Any subclass of init should call $self->SUPER::init().

create_table( $table )

Create a table named $table.

get( $table, $key )

Get the $key entry in the $table table.

put( $table, $key, $val )

Set the $key entry in the $table table to the value $val.

del( $table, $key )

Delete the $key entry from the $table table.

get_keys( $table )

Delete a list of the keys from the $table table.

nkeys( $table )

Returns the number of keys in the $table table.

SEE ALSO

HTML::Index

HTML::Index::Store::BerkeleyDB

HTML::Index::Store::DataDumper

Compress::Zlib

Lingua::Stem

Text::Soundex

AUTHOR

Ave Wrigley <Ave.Wrigley@itn.co.uk>

COPYRIGHT

Copyright (c) 2003 Ave Wrigley. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.