NAME

WWW::Mixi::Scraper::Plugin - base class for plugins

SYNOPSIS

    package WWW::Mixi::Scraper::Plugin::<SamplePlugin>

    use strict;
    use warnings;
    use WWW::Mixi::Scraper::Plugin;

    validator {qw( id is_number )};

    sub scrape {
      my ($self, $html) = @_;

      my %scraper;
      $scraper{...} = scraper {
        process '...',
          text => 'TEXT';  # always plain text
        process '...',
          text => 'HTML';  # always HTML
        process '...',
          depends => $self->html_or_text; # HTML or TEXT
        result qw( text depends );
      };

      return $self->post_process($scraper{...}->scrape(\$html));

      return $arrayref;
    }

    1;

DESCRIPTION

This is a base class for WWW::Mixi::Scraper plugins. You don't need to use base qw( WWW::Mixi::Scraper::Plugin ); just use it. This exports Web::Scraper functions and several public and private methods/functions of its own.

METHODS

new

creates an object.

html_or_text

returns the rendering mode of choice ('TEXT' or 'HTML').

parse

gets content from a uri, scrapes, and returns an array (or a hash reference, etc) of data.

build_uri

used internally to build uri from query paramters (and optional hash). If you want to tweak the uri (e.g. to change authority from 'mixi.jp' to 'video.mixi.jp', etc), provide tweak_uri() in your plugin.

get_content

gets content from the uri specified or from an optional html data.

post_process

does some common tasks such as to make sure the result be an array reference, remove some temporary data to extract exact information, objectify link data (and maybe datetime data in the future?).

validator

prepares some simple validator for query parameters, though I'm not sure if this is really useful.

AUTHOR

Kenichi Ishigaki, <ishigaki at cpan.org>

COPYRIGHT AND LICENSE

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

CPAN RT