Logan Bell > Lucy-0.3.2 > Lucy::Analysis::SnowballStopFilter

Download:
Lucy-0.3.2.tar.gz

Dependencies

Annotate this POD

CPAN RT

New  3
Open  0
View/Report Bugs
Module Version: 0.003002   Source   Latest Release: Lucy-0.3.3

NAME ^

Lucy::Analysis::SnowballStopFilter - Suppress a "stoplist" of common words.

SYNOPSIS ^

    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        language => 'fr',
    );
    my $polyanalyzer = Lucy::Analysis::PolyAnalyzer->new(
        analyzers => [ $case_folder, $tokenizer, $stopfilter, $stemmer ],
    );

DESCRIPTION ^

A "stoplist" is collection of "stopwords": words which are common enough to be of little value when determining search results. For example, so many documents in English contain "the", "if", and "maybe" that it may improve both performance and relevance to block them.

Before filtering stopwords:

    ("i", "am", "the", "walrus")

After filtering stopwords:

    ("walrus")

SnowballStopFilter provides default stoplists for several languages, courtesy of the Snowball project (<http://snowball.tartarus.org>), or you may supply your own.

    |-----------------------|
    | ISO CODE | LANGUAGE   |
    |-----------------------|
    | da       | Danish     |
    | de       | German     |
    | en       | English    |
    | es       | Spanish    |
    | fi       | Finnish    |
    | fr       | French     |
    | hu       | Hungarian  |
    | it       | Italian    |
    | nl       | Dutch      |
    | no       | Norwegian  |
    | pt       | Portuguese |
    | sv       | Swedish    |
    | ru       | Russian    |
    |-----------------------|

CONSTRUCTORS ^

new( [labeled params] )

    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        language => 'de',
    );
    
    # or...
    my $stopfilter = Lucy::Analysis::SnowballStopFilter->new(
        stoplist => \%stoplist,
    );

INHERITANCE ^

Lucy::Analysis::SnowballStopFilter isa Lucy::Analysis::Analyzer isa Lucy::Object::Obj.

syntax highlighting: