Search results for "WWW::RobotRules::AnyDBM_File"
WWW::RobotRules::AnyDBM_File - Persistent RobotRules
This is a subclass of *WWW::RobotRules* that uses the AnyDBM_File package to implement persistent diskcaching of robots.txt and host visit information. The constructor (the new() method) takes an extra argument specifying the name of the DBM file to ...
GAAS/WWW-RobotRules-6.02 - 18 Feb 2012 13:09:13 UTC - Search in distribution- WWW::RobotRules - database of robots.txt-derived permissions
LWP - The World-Wide Web library for Perl
The libwww-perl collection is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web. The main focus of the library is to provide classes and functions that allow you to write WWW cl...
OALDERS/libwww-perl-6.77 - 11 Mar 2024 00:58:25 UTC - Search in distribution
LWP::Parallel - Extension for LWP to allow parallel HTTP and FTP access
Introduction ParallelUserAgent is an extension to the existing libwww module. It allows you to take a list of URLs (it currently supports HTTP, FTP, and FILE URLs. HTTPS might work, too) and connect to all of them _in parallel_, then wait for the res...
MSOUTH/ParallelUserAgent-2.62 - 29 May 2016 18:55:38 UTC - Search in distribution- LWP::Parallel::RobotUA - A class for Parallel Web Robots
WWW::RobotRules::Extended - database of robots.txt-derived permissions. This is a fork of WWW::RobotRules
This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> It also parses rules that contains wildcards '*' and allow directives like Google does. Webmasters can use the /robo...
YSIMONX/WWW-RobotRules-Extended-0.02 - 14 Jan 2012 10:23:47 UTC - Search in distribution
Bundle::FinalTest - Perl extension for blah blah blah
Stub documentation for Bundle::FinalTest, created by h2xs. It looks like the author of the extension was negligent enough to leave the stub unedited. Blah blah blah....
MARSAB/Bundle-FinalTest - 02 Aug 2011 05:52:05 UTC - Search in distribution
Bundle::FinalTest2 - Perl extension for blah blah blah
Stub documentation for Bundle::FinalTest2, created by h2xs. It looks like the author of the extension was negligent enough to leave the stub unedited....
MARSAB/Bundle-FinalTest2 - 03 Aug 2011 05:10:02 UTC - Search in distribution