Treex::Block::W2A::SegmentOnNewlines - segment text on new lines
version 0.08170
The source text is segmented into sentences which are stored in document bundles. If the document contained no bundles, the bundles are created. Otherwise, the document must contain the same number of bundles as the number of sentences (segmented by this blocks). This means that this block (or its derivatives) can be used in this way:
treex Read::Text language=en from=en.txt \ Read::Text language=de from=de.txt \ W2A::SegmentOnNewlines language=en \ W2A::SegmentOnNewlines language=de \ ...
This class detects sentences based on the newlines in the source text, but it can be used as an ancestor for more apropriate segmentations by overriding the method segment_text.
segment_text
If set, empty sentences can be produced.
If set, empty sentences are automatically deleted.
If none of the previous attributes is set and empty sentence found, fatal is raised.
This method produces segments from text given as parameter, can be overriden.
This method does sentence normalization e.g. trims initial and terminal whitespaces.
Martin Popel <popel@ufal.mff.cuni.cz>
Copyright © 2011 by Institute of Formal and Applied Linguistics, Charles University in Prague
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install Treex::Unilang, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Treex::Unilang
CPAN shell
perl -MCPAN -e shell install Treex::Unilang
For more information on module installation, please visit the detailed CPAN module installation guide.