Daisuke Maki > HTTP-RobotsTag-0.00001 > HTTP::RobotsTag

Download:
HTTP-RobotsTag-0.00001.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Module Version: 0.00001   Source  

NAME ^

HTTP::RobotsTag - Parse Robots Tag In HTTP Headers

SYNOPSIS ^

  use HTTP::RobotsTag;

  my $response = $lwp->get( $url );
  my $p        = HTTP::RobotsTag->new();
  my $rule     = $p->parse_headers( $response );

  if ($rule->can_index()) {
    ...
  }

  if ($rule->is_available( $dt )) {
    ...
  }

DESCRIPTION ^

HTTP::RobotsTag parses HTTP headers for X-Robots-Tag headers and stores the information in a Rules object.

METHODS ^

new

parse_headers($headers)

Accepts a HTTP::Headers (or an object that implements the header()) method, and looks for X-Robots-Tag headers.

AUTHOR ^

Copyright (c) 2007 Daisuke Maki <daisuke@endeworks.jp>

SEE ALSO ^

HTML::RobotsTag::Rules

LICENSE ^

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://www.perl.com/perl/misc/Artistic.html

syntax highlighting: