Xango-1.08_01 - Event Based High Performance Web Crawler Framework - metacpan.org

17 May 2006 09:05:56 UTC
Development release
Browse (raw)
Changes
How to Contribute
Issues (1)
Testers (8 / 8 / 0)
Kwalitee
License: perl_5
Activity
24 month
Tools
Download (29.12KB)
MetaCPAN Explorer
Permissions
Subscribe to distribution
- Dependencies
- Cache::Cache
- Data::Average
- HTTP::Request
- HTTP::Response
- Log::Dispatch
- POE
- YAML
- and possibly others
- Reverse dependencies
- CPAN Testers List
- Dependency graph
Permalinks
This version
- Copyright 2005-2006 Daisuke Maki E<lt>dmaki@cpan.orgE<gt>. All rights reserved. Development funded by Brazil, Ltd. E<lt>http://b.razil.jpE<gt>

Daisuke Maki /

Xango-1.08_01

Changes for version 1.08_01

Add clear() call to the DNS cache on startup. On systems with prolonged process life time, it's likely that the DNS cache is holding bad data, so we might as well reinitialize.
add tools/checkreqs.pl and custom Makefile.PL
Fix Push.pm so that shutdown_broker state kills alarms.
EXPERIMETAL
- Try to work with servers that send 400/408 upon too many connections (the sysadmins must hate me) by gracefully reducing the number of HTTP fetchers upon encountering those return status

Documentation

Xango::Manual::Intro

Learn How To Write Crawlers With Xango

Modules

Xango

Event Based High Performance Web Crawler Framework

Broker HTTP Requests (Deprecated)

Xango::Broker::Base

Base Class for Xango Broker

Xango::Broker::Pull

Xango's Pull-Crawler

Xango::Broker::Push

Xango's Push-Crawler

Global Xango Config

Xango Job

Other files

To install Xango, copy and paste the appropriate command in to your terminal.

cpanm Xango

perl -MCPAN -e shell
install Xango

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)