NAME
Lingua::EN::Summarize - A simple tool for summarizing bodies of English
text.
SYNOPSIS
use Lingua::EN::Summarize;
my $summary = summarize( $text ); # Easy, no? :-)
my $summary = summarize( $text, maxlength => 500 ); # 500-byte summary
my $summary = summarize( $text, filter => 'html' ); # Strip HTML formatting
my $summary = summarize( $text, wrap => 75 ); # Wrap output to 75 col.
DESCRIPTION
This is a simple module which makes an unscientific effort at
summarizing English text. It recognizes simple patterns which look like
statements, abridges them, and concatenates them into something vaguely
resembling a summary. It needs more work on large bodies of text, but it
seems to have a decent effect on small inputs at the moment.
Lingua::EN::Summarize exports one function, "summarize()", which takes
the text to summarize as its first argument, and any number of optional
directives in "name => value" form. The options it'll take are:
maxlength
Specifies the maximum length, in bytes, of the generated summary.
wrap
Prettyprints the summary output by wrapping it to the number of
columns which you specify.
filter
Passes the text through a filter before handing it to the
summarizer. Currently, only two filters are implemented: ""html"",
which uses HTML::TreeBuilder and HTML::FormatText to strip all HTML
formatting from a document, and ""easyhtml"", which quickly (and
less accurately) strips all HTML from a document using a simple
regular expression, if you don't have the abovementioned modules. An
""email"" filter, for converting mail and news messages to
easily-summarizable text, is in the works for the next version.
Unlike the HTML::Summarize module (which is very cool, and worth a
look), this module considers its input to be plain English text, and
doesn't try to gather any information from the formatting. Thus, without
any cues from the document's format, the scheme that HTML::Summarize
uses isn't applicable here. The current scheme goes something like this:
"Filter the text according to the user's "filter" option. Split the text
into discrete sentences with the Text::Sentence module, then further
split them into clauses on commas and semicolons. Keep only the ones
that have a (subject very-simple-verb object) structure. Construct the
summary out of the first sentences in the list, staying within the
"maxlength" limit, or under 30% of the size of the original text,
whichever is smaller."
Needless to say, this is a very simple and not terribly universally
effective scheme, but it's good enough for a first draft, and I'll bang
on it more later. Like I said, it's not a scientific approach to the
problem, but it's better than nothing, and I don't really need A.I.
quality output from it.
AUTHOR
Dennis Taylor, <dennis@funkplanet.com>
SEE ALSO
HTML::Summarize, Text::Sentence,
http://www.vancouvertoday.com/city_guide/dining/reviews/barbers_modern_c
lub.html