View on
MetaCPAN
search.cpan.org is shutting down
For details read Perl NOC. After June 25th this page will redirect to MetaCPAN.org
Tom Gracey > TaskPipe-0.03

TaskPipe-0.03

This Release TaskPipe-0.03  [Download] [Browse 16 May 2018
Other Releases
Links Discussion Forum ] [ View/Report Bugs (1) ] [ Dependencies ] [ Other Tools ]
CPAN Testers PASS (87)   [ View Reports ] [ Perl/Platform Version Matrix ]
Rating      (0 Reviews) [ Rate this distribution ]
License The GNU General Public License, Version 3, June 2007
Special Files
Changes
Makefile.PL
MANIFEST
META.json
README

Modules

TaskPipe A task management framework for building web scrapers and crawlers      
TaskPipe::DaemonManager manage TaskPipe background processes      
TaskPipe::FileInstaller handles file installs for TaskPipe      
TaskPipe::InterpParam Parameter value interpolator for TaskPipe      
TaskPipe::InterpParam::MatchHandler handling matching of parameter values in the plan      
TaskPipe::InterpParam::MatchHandler_id Match handler for the $id parameter variable      
TaskPipe::InterpParam::MatchHandler_name match handler for the $name parameter variable      
TaskPipe::InterpParam::MatchHandler_this match handler for the $this parameter variable      
TaskPipe::InterpParam::Parts module determining the parts of a plan parameter variable      
TaskPipe::Iterator generic class for TaskPipe Iterators      
TaskPipe::JobManager manages TaskPipe jobs      
TaskPipe::JobManager::Settings settings for the TaskPipe::JobManager module      
TaskPipe::LoggerManager Logging manager for TaskPipe      
TaskPipe::LoggerManager::Settings settings for the TaskPipe::LoggerManager module      
TaskPipe::OpenProxyManager open proxy management for TaskPipe      
TaskPipe::OpenProxyManager::IPList the base class for IPLists      
TaskPipe::OpenProxyManager::IPList::Settings Settings for TaskPipe::OpenProxyManager::IPList      
TaskPipe::OpenProxyManager::IPList_PremProxy PremProxy IP List      
TaskPipe::OpenProxyManager::IPList_PremProxy::Settings settings for TaskPipe::OpenProxyManager::IPList_PremProxy      
TaskPipe::OpenProxyManager::IPList_ProxyNova ProxyNova IP List      
TaskPipe::OpenProxyManager::IPList_ProxyNova::Settings Settings for TaskPipe::OpenProxyManager::IPList_ProxyNova      
TaskPipe::OpenProxyManager::IPList_Xroxy Xroxy IP List      
TaskPipe::OpenProxyManager::IPList_Xroxy::Settings settings for TaskPipe::OpenProxyManager::IPList_Xroxy      
TaskPipe::OpenProxyManager::Settings settings for TaskPipe::OpenProxyManager      
TaskPipe::PathSettings Path settings for TaskPipe      
TaskPipe::PathSettings::Global TaskPipe global path settings      
TaskPipe::PathSettings::Project project path settings for TaskPipe      
TaskPipe::Plan manage plan files for TaskPipe      
TaskPipe::Plan::Settings settings for TaskPipe::Plan      
TaskPipe::PodReader read POD for TaskPipe      
TaskPipe::PodReader::Settings Settings for TaskPipe::PodReader      
TaskPipe::PortManager manage ports for TaskPipe      
TaskPipe::Role::MooseType_IterateMode iterate mode type constraint      
TaskPipe::Role::MooseType_OutputMode output mode type constraint      
TaskPipe::Role::MooseType_PlanMode plan mode type constraint      
TaskPipe::Role::MooseType_ScopeMode scope mode type constraint      
TaskPipe::Role::MooseType_ShellMode shell mode type constraint      
TaskPipe::Role::MooseType_StrArrayRef array ref represented as a string type constraint      
TaskPipe::Role::MooseType_UserAgentType user agent type type constraint      
TaskPipe::RunInfo information about the run (of the current plan)      
TaskPipe::Sample base class for sample projects      
TaskPipe::Sample_SP500 sample project to scrape quotes for S&P500 companies      
TaskPipe::Sample_Stubs the default sample that is used when deploying files/db tables      
TaskPipe::SchemaManager manage schema connections for TaskPipe      
TaskPipe::SchemaManager::Settings settings for TaskPipe::SchemaManager      
TaskPipe::SchemaManager::Settings_Global global schema settings for TaskPipe      
TaskPipe::SchemaManager::Settings_Project project schema settings for TaskPipe      
TaskPipe::SchemaTemplate_Global Global Schema Template for TaskPipe      
TaskPipe::SchemaTemplate_Global::Result::Daemon schema template for the global daemon table      
TaskPipe::SchemaTemplate_Global::Result::Job global schema template for job source      
TaskPipe::SchemaTemplate_Global::Result::OpenProxy global schema template for open proxy source      
TaskPipe::SchemaTemplate_Global::Result::Port Global Schema Template file for port source      
TaskPipe::SchemaTemplate_Global::Result::Spawned global schema template for spawned source      
TaskPipe::SchemaTemplate_Global::Result::Thread global schema template for thread source      
TaskPipe::SchemaTemplate_Project Project Schema Template for TaskPipe      
TaskPipe::SchemaTemplate_Project::Result::Error schema template for the project error table      
TaskPipe::SchemaTemplate_Project::Result::Pinterp schema template for the project pinterp table      
TaskPipe::SchemaTemplate_Project::Result::Result schema template for the project result table      
TaskPipe::SchemaTemplate_Project::Result::ResultGroup schema template for the project result_group table      
TaskPipe::SchemaTemplate_Project::Result::Run schema template for project run table      
TaskPipe::SchemaTemplate_Project::Result::XBranchError schema template for project xbranch_error table      
TaskPipe::SchemaTemplate_Project::Result::Xbranch schema template for project xbranch table      
TaskPipe::SchemaTemplate_Project_SP500 schema template for the SP500 sample project      
TaskPipe::SchemaTemplate_Project_SP500::Result::Company schema template for the SP500 project company table      
TaskPipe::Task the class which all TaskPipe tasks should inherit from     0.03
TaskPipe::Task::ModuleMap map task names to modules      
TaskPipe::Task::Settings Settings for TaskPipe::Task      
TaskPipe::Task::TestSettings test settings for TaskPipe::Task      
TaskPipe::Task_Record record a record to the database      
TaskPipe::Task_Record_NameValuePairs record a result set as a series of name/value pairs      
TaskPipe::Task_Scrape Base TaskPipe class for scraping a webpage      
TaskPipe::Task_Scrape::Settings settings for TaskPipe::Task_Scrape      
TaskPipe::Task_SourceFromDB use a database table as the data source      
TaskPipe::Task_SourceFromFile use a file as a data source      
TaskPipe::Template the base class for file templates      
TaskPipe::Template_Config      
TaskPipe::Template_Config_Global template for the global config file      
TaskPipe::Template_Config_Project template for the project config      
TaskPipe::Template_Config_Project_SP500      
TaskPipe::Template_Config_System the template package for the 'system' config file      
TaskPipe::Template_Plan the base class for plan templates      
TaskPipe::Template_Plan_SP500 template for the SP500 sample project plan      
TaskPipe::Template_Plan_Stub template for the default plan      
TaskPipe::Template_Task base class for task templates      
TaskPipe::Template_Task_SP500ScrapeCompanies      
TaskPipe::Template_Task_SP500ScrapeQuote template for the Scrape_Quote task in the SP500 sample project      
TaskPipe::Template_Task_ScrapeStub template for the Scrape_Stub task      
TaskPipe::ThreadManager manages threads for TaskPipe      
TaskPipe::Tool the base class for the TaskPipe command line tool     0.03
TaskPipe::Tool::Command base class for TaskPipe Tool commands      
TaskPipe::Tool::Command_ClearTables command to clear tables      
TaskPipe::Tool::Command_DeployFiles command to deploy TaskPipe project files      
TaskPipe::Tool::Command_DeployTables command to deploy taskpipe tables      
TaskPipe::Tool::Command_GenerateSchema command to generate taskpipe schema files      
TaskPipe::Tool::Command_RunPlan command to run a TaskPipe plan      
TaskPipe::Tool::Command_Setup the TaskPipe setup command      
TaskPipe::Tool::Command_ShowJobs show currently running TaskPipe jobs      
TaskPipe::Tool::Command_StopJob command to stop a running taskpipe job      
TaskPipe::Tool::Command_TestTask command to test an individual TaskPipe task      
TaskPipe::Tool::Command__OpenProxies base calss for open proxy related commands      
TaskPipe::Tool::Command__OpenProxies_FetchOpenProxies command to fetch open proxies      
TaskPipe::Tool::Command__OpenProxies_TestOpenProxies command to test open proxies      
TaskPipe::Tool::Options handles command line options for TaskPipe Tool      
TaskPipe::TorManager manage TOR processes for TaskPipe      
TaskPipe::TorManager::Settings settings for TaskPipe::TorManager      
TaskPipe::UserAgentManager base class for managing requests in TaskPipe      
TaskPipe::UserAgentManager::CheckIPSettings      
TaskPipe::UserAgentManager::Settings settings for TaskPipe::UserAgentManger      
TaskPipe::UserAgentManager::UserAgentHandler standard useragent handler      
TaskPipe::UserAgentManager::UserAgentHandler::Settings      
TaskPipe::UserAgentManager::UserAgentHandler_PhantomJS useragent handler for phantomjs      
TaskPipe::UserAgentManager::UserAgentHandler_PhantomJS::Settings      
TaskPipe::UserAgentManager::UserAgentHandler_WhoIs      
TaskPipe::UserAgentManager_ProxyNet base class for proxying useragents      
TaskPipe::UserAgentManager_ProxyNet_Open useragent manager for making requests through an open proxy network      
TaskPipe::UserAgentManager_ProxyNet_TOR useragent manager for making requests through TOR      

Documentation

TaskPipe::Manual::Installation how to install and setup TaskPipe  
TaskPipe::Manual::Overview An overview of the TaskPipe framework  
TaskPipe::Template_Task_SP500Scrape_Companies template for the Scrape_Companies task in the SP500 sample project  
taskpipe script to execute the TaskPipe Tool