Search results for "robots.txt"
Logfile::EPrints::Filter::RobotsTxt - Filter Web log hits using a database of robot's IPs
TIMBRODY/Logfile-EPrints-1.20
-
14 Sep 2009 11:40:23 UTC
-
Search in distribution
WWW::RobotRules - database of robots.txt-derived permissions
This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters can use the /robots.txt file to forbid conforming robots from accessing parts of their web site. The pars...
GAAS/WWW-RobotRules-6.02 - 18 Feb 2012 13:09:13 UTC - Search in distribution- WWW::RobotRules::AnyDBM_File - Persistent RobotRules
phoebe - a Gemini-first wiki server
Phoebe does two and a half things: It's a program that you run on a computer and other people connect to it using their Gemini client in order to read the pages on it. It's a wiki, which means that people can edit the pages without needing an account...
SCHROEDER/App-Phoebe-4.07 - 21 May 2023 13:34:48 UTC - Search in distribution- App::Phoebe::SpeedBump - defend Phoebe against bots and leeches
Apache::ASP - Active Server Pages for Apache with mod_perl
Apache::ASP provides an Active Server Pages port to the Apache Web Server with Perl scripting only, and enables developing of dynamic web applications with session management and embedded Perl code. There are also many powerful extensions, including ...
CHAMAS/Apache-ASP-2.63 - 15 Mar 2018 05:28:37 UTC - Search in distribution
LWP - The World-Wide Web library for Perl
The libwww-perl collection is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web. The main focus of the library is to provide classes and functions that allow you to write WWW cl...
OALDERS/libwww-perl-6.77 - 11 Mar 2024 00:58:25 UTC - Search in distribution- lwptut - An LWP Tutorial
- LWP::RobotUA - a class for well-behaved Web robots
MVC::Neaf - Not Even A (Web Application) Framework
KHEDIN/MVC-Neaf-0.2901
-
02 Aug 2023 00:28:03 UTC
-
Search in distribution
perlperf - Perl Performance and Optimization Techniques
This is an introduction to the use of performance and optimization techniques which can be used with particular reference to perl programs. While many perl developers have come from other languages, and can use their prior knowledge where appropriate...
PEVANS/perl-5.38.2 - 29 Nov 2023 16:10:36 UTC - Search in distribution- perlperf - Perl Performance and Optimization Techniques
robots
MIKEDLR/Link_Controller-0.037
-
09 Feb 2002 18:12:34 UTC
-
Search in distribution
- ElephantAgent - the agent that never forgets
Scaffold - Web Application Infrastructure
KESTEB/Scaffold-0.10
-
27 Apr 2012 03:57:20 UTC
-
Search in distribution
- Scaffold::Server - The Scaffold web engine
- Scaffold::Engine - The Scaffold interface to Plack/psgi
- Scaffold::Handler::Static - A handler for static files
- 5 more results from Scaffold »
Apache::AppSamurai - An Authenticating Mod_Perl Front End
Apache::AppSamurai protects web applications from direct attack by unauthenticated users, and adds a flexible authentication front end to local or proxied applications with limited authentication options. Unauthenticated users are presented with eith...
PAULDOOM/Apache-AppSamurai-1.01 - 03 May 2008 07:10:21 UTC - Search in distribution
Gungho.ja - 高性能Webクローラーフレームワーク
Gunghoは高性能Webクローラーフレームワークです。高速なHTTP処理を行いつつ、 機能拡張をしやすいようフレキシブルな構造を目指して開発されています。 現在Gunghoはβ版です。機能的/仕様的にも比較的安定しつつありますが、まだ 内部的なAPI等は大幅な変更が加わる可能性がありますのでご注意ください。 Gunghoをインストールすると自動的に以下の機能が使えるようになります: イベント型非同期エンジン GunghoはPOE、Danga::Socket、IO::Async等をベースにした非...
DMAKI/Gungho-0.09008 - 28 Jul 2008 10:37:52 UTC - Search in distribution- Gungho - Yet Another High Performance Web Crawler Framework
- Gungho::Manual::FAQ.ja - Gungho FAQ
- Gungho::Manual::Intro.ja - 高性能クローラーフレームワーク
- 7 more results from Gungho »
Mojolicious::Guides::Cookbook - Cooking with Mojolicious
SRI/Mojolicious-9.37
-
13 May 2024 17:48:56 UTC
-
Search in distribution
LWP::Parallel - Extension for LWP to allow parallel HTTP and FTP access
Introduction ParallelUserAgent is an extension to the existing libwww module. It allows you to take a list of URLs (it currently supports HTTP, FTP, and FILE URLs. HTTPS might work, too) and connect to all of them _in parallel_, then wait for the res...
MSOUTH/ParallelUserAgent-2.62 - 29 May 2016 18:55:38 UTC - Search in distribution- LWP::Parallel::RobotUA - A class for Parallel Web Robots
WWW::GoKGS - KGS Go Server (http://www.gokgs.com/) Scraper
This module is a KGS Go Server ("http://www.gokgs.com/") scraper. KGS allows the users to play a board game called go a.k.a. baduk (Korean) or weiqi (Chinese). Although the web server provides resources generated dynamically, such as Game Archives, t...
ANAZAWA/WWW-GoKGS-0.21 - 21 Aug 2014 02:27:48 UTC - Search in distribution- WWW::GoKGS::Scraper::GameArchives - KGS Game Archives
Search::Circa - a Search Engine / Indexer running with Mysql
This is Search::Circa, a module who provide functions to perform search on Circa, a www search engine running with Mysql. Circa is for your Web site, or for a list of sites. It indexes like Altavista does. It can read, add and parse all url's found i...
ALIAN/Search-Circa-1.18 - 02 Jan 2003 12:35:27 UTC - Search in distribution- Search::Circa::Indexer - provide functions to administrate Circa, a www search engine running with Mysql
File::HTTP - open, read and seek into remote files and directories transparently
"File::HTTP" open, read and seek into remote files and directories transparently open [MODE] FILE stat FH or FILE Imported with the :open tag. Act exaclty as CORE::open and CORE::stat, but also work with remote HTTP files. Falls back to CORE::open an...
TDRUGEON/File-HTTP-1.11 - 17 Mar 2023 23:02:11 UTC - Search in distribution
Statocles::Site - An entire, configured website
A Statocles::Site is a collection of applications....
PREACTION/Statocles-0.098 - 11 Apr 2021 19:09:54 UTC - Search in distribution
serve_file - Plugin for serving raw files
This plugin turns AxKit2 into a normal every-day httpd. Yay! Most httpds need to serve plain files. Things like favicon.ico and robots.txt that any sane web server would be lost without. So just load this plugin after all the others, and if your othe...
MSERGEANT/AxKit2-1.1 - 09 Sep 2006 21:26:43 UTC - Search in distribution- serve_file - Plugin for serving raw files