HTML::FormatExternal - HTML to text formatting using external programs
This is a collection of formatter modules turning HTML into plain text by dumping it through the respective external programs.
HTML::FormatText::Elinks HTML::FormatText::Html2text HTML::FormatText::Links HTML::FormatText::Lynx HTML::FormatText::Netrik HTML::FormatText::Vilistextum HTML::FormatText::W3m HTML::FormatText::Zen
The module interfaces are compatible with
HTML::Formatter modules such as
HTML::FormatText, but the external programs do all the work.
Common formatting options are used where possible, such as
rightmargin. So just by switching the class you can use a different program (or the plain
HTML::FormatText) according to personal preference, or strengths and weaknesses, or what you've got.
There's nothing particularly difficult about piping through these programs, but a unified interface hides details like how to set margins and how to force input or output charsets.
Each of the classes above provide the following functions. The
XXX in the class names here is a placeholder for any of
Lynx, etc as above.
See examples/demo.pl in the HTML-FormatExternal sources for a complete sample program.
$text = HTML::FormatText::XXX->format_file ($filename, key=>value,...)
$text = HTML::FormatText::XXX->format_string ($html_string, key=>value,...)
Run the formatter program over a file or string with the given options and return the formatted result as a string. See "OPTIONS" below for possible key/value options. For example,
$text = HTML::FormatText::Lynx->format_file ('/my/file.html'); $text = HTML::FormatText::W3m->format_string ('<html><body> <p> Hello world! </p </body></html>');
For reference, it might be noted some of the programs interpret command line names like "-" as standard input, or "http:" as a url. The way
HTML::FormatExternal runs them ensures any
$filename given to
format_file() is taken literally. So for example passing "-" reads a file called "-".
$formatter = HTML::FormatText::XXX->new (key=>value, ...)
Create a formatter object with the given options. In the current implementation an object doesn't do much more than remember the options for future use.
$formatter = HTML::FormatText::Elinks->new(rightmargin => 60);
$text = $formatter->format ($tree_or_string)
$formatter program on a
HTML::TreeBuilder tree or a string, using the options in
$formatter, and return the result as a string.
A TreeBuilder argument (ie. a
HTML::Element) is accepted for compatibility with
HTML::Formatter. The tree is simply turned into a string with
$tree->as_HTML to pass to the program, so if you've got a string already then give that instead of a tree.
HTML::Element itself has a
format() method (see "format" in HTML::Element) which runs a given
HTML::FormatExternal can be used for
$text = $tree->format($formatter); # which dispatches to $text = $formatter->format($tree);
The following are extra methods not available in the plain
Return the version number of the formatter program as reported by its
--version or similar option. If the formatter program is not available then return
program_version() is the bare version number, though perhaps with "beta" or similar indication.
program_full_version() is the entire version output, which may include build options, copyright notice, etc.
$str = HTML::FormatText::Lynx->program_version(); # eg. "2.8.7dev.10" $str = HTML::FormatText::W3m->program_full_version(); # eg. "w3m version w3m/0.5.2, options lang=en,m17n,image,..."
The version number of the Perl module itself is available in the usual way (see "VERSION" in UNIVERSAL).
$modulever = HTML::FormatText::Netrik->VERSION; $modulever = $formatter->VERSION
A file passed to the formatter programs is interpreted by them in the charset of a
<meta> within the HTML, or default latin-1 per the HTML specs, or as forced by the
input_charset option below.
A string input should be bytes the same as a file, not Perl wide chars. (There's some secret experimental encode/decode for wide chars, as yet unused, better let
HTML::Formatter take the lead on how that might be activated.)
The result string is bytes similarly, encoded in whatever the respective programs produce. This may be the locale charset or you can force it with the
output_charset option to be sure.
The following options can be given. The defaults are whatever the respective programs do. The programs generally read their config files when dumping so the defaults and formatting details might follow your personal settings (usually a good thing).
leftmargin => INTEGER
rightmargin => INTEGER
The column numbers for the left and right hand ends of the text.
leftmargin 0 means no padding on the left.
rightmargin is the text width, so for instance 60 would mean the longest line is 60 characters (inclusive of any
leftmargin). These options are compatible with
rightmargin is not necessarily a hard limit. Some of the programs will exceed it in a HTML literal
<pre>, or a run of
input_charset => STRING
Force the HTML input to be interpreted as bytes of the given charset, including ignoring any
<meta> within the HTML.
output_charset => STRING
Force the text output to be encoded as the given charset. The default varies among the programs, but usually defaults to the locale.
base => STRING
Set the base URL for any relative links within the HTML (similar to
HTML::FormatText::WithLinks). Usually this should be the location the HTML was downloaded from.
If the document contains its own
<base> setting then currently the document takes precedence. Only Lynx and Elinks display absolutized link targets and option has no effect on the other programs.
There's nothing done with errors or warning messages from the formatters. Generally they make a best effort on doubtful HTML, but fatal errors like bad options or missing libraries should be trapped in the future.
elinks (from Aug 2008 onwards) and
netrik can produce ANSI escapes for colours, underline, etc, and
html2text can produce tty style backspace overstriking. This might be good for text destined for a tty or further crunching. Perhaps an
tty option could enable this, where possible, but for now it's deliberately turned off in those programs to keep the default as plain text.
HTML::FormatText::Elinks, HTML::FormatText::Html2text, HTML::FormatText::Links, HTML::FormatText::Netrik, HTML::FormatText::Lynx, HTML::FormatText::Vilistextum, HTML::FormatText::W3m, HTML::FormatText::Zen
Copyright 2008, 2009, 2010, 2011, 2012, 2013 Kevin Ryde
HTML-FormatExternal is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version.
HTML-FormatExternal is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with HTML-FormatExternal. If not, see <http://www.gnu.org/licenses/>.