Kenichi Ishigaki > WWW-Mixi-Scraper-0.34 > WWW::Mixi::Scraper::Plugin

Download:
WWW-Mixi-Scraper-0.34.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Source  

NAME ^

WWW::Mixi::Scraper::Plugin - base class for plugins

SYNOPSIS ^

    package WWW::Mixi::Scraper::Plugin::<SamplePlugin>

    use strict;
    use warnings;
    use WWW::Mixi::Scraper::Plugin;

    validator {qw( id is_number )};

    sub scrape {
      my ($self, $html) = @_;

      my %scraper;
      $scraper{...} = scraper {
        process '...',
          text => 'TEXT';  # always plain text
        process '...',
          text => 'HTML';  # always HTML
        process '...',
          depends => $self->html_or_text; # HTML or TEXT
        result qw( text depends );
      };

      return $self->post_process($scraper{...}->scrape(\$html));

      return $arrayref;
    }

    1;

DESCRIPTION ^

This is a base class for WWW::Mixi::Scraper plugins. You don't need to use base qw( WWW::Mixi::Scraper::Plugin ); just use it. This exports Web::Scraper functions and several public and private methods/functions of its own.

METHODS ^

new

creates an object.

html_or_text

returns the rendering mode of choice ('TEXT' or 'HTML').

parse

gets content from a uri, scrapes, and returns an array (or a hash reference, etc) of data.

build_uri

used internally to build uri from query paramters (and optional hash). If you want to tweak the uri (e.g. to change authority from 'mixi.jp' to 'video.mixi.jp', etc), provide tweak_uri() in your plugin.

get_content

gets content from the uri specified or from an optional html data.

post_process

does some common tasks such as to make sure the result be an array reference, remove some temporary data to extract exact information, objectify link data (and maybe datetime data in the future?).

validator

prepares some simple validator for query parameters, though I'm not sure if this is really useful.

AUTHOR ^

Kenichi Ishigaki, <ishigaki at cpan.org>

COPYRIGHT AND LICENSE ^

Copyright (C) 2007 by Kenichi Ishigaki.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

syntax highlighting: