Daisuke Maki > HTTP-RobotsTag > HTTP::RobotsTag::Rules

Download:
HTTP-RobotsTag-0.00001.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Source  

NAME ^

HTTP::RobotsTag::Rules - A Representation Of Robots Exclusion Rules

SYNOPSIS ^

  $rules = $p->parse_rules($html);
  $bool  = $rules->can_index();
  $bool  = $rules->can_follow();
  $bool  = $rules->can_archive();
  $bool  = $rules->can_snippet();
  $bool  = $rules->is_available( $dt );

DESCRIPTION ^

HTTP::RobotsTag::Rules represents the robots exclusion policies that are described within HTTP headers.

METHODS ^

new

can_index

can_follow

can_archive

can_snippet

is_available($dt)

Returns true if the resource is available on $dt.

syntax highlighting: