Mario Roy > MCE > MCE

Download:
MCE-1.608.tar.gz

Dependencies

Annotate this POD

Website

CPAN RT

Open  3
View/Report Bugs
Module Version: 1.608   Source  

NAME ^

MCE - Many-Core Engine for Perl providing parallel processing capabilities

VERSION ^

This document describes MCE version 1.608

DESCRIPTION ^

Many-Core Engine (MCE) for Perl helps enable a new level of performance by maximizing all available cores. MCE spawns a pool of workers and therefore does not fork a new process per each element of data. Instead, MCE follows a bank queuing model. Imagine the line being the data and bank-tellers the parallel workers. MCE enhances that model by adding the ability to chunk the next n elements from the input stream to the next available worker.

Input data is optional in MCE. Thus, input data is not required to run MCE.

CORE MODULES ^

Three modules make up the core engine for MCE.

MCE::Core

Provides the Core API for Many-Core Engine.

MCE::Signal

Temporary directory creation/cleanup and signal handling.

MCE::Util

Utility functions for Many-Core Engine.

MCE EXTRAS ^

There are 3 add-on modules.

MCE::Mutex

Provides a simple semaphore implementation supporting threads and processes.

MCE::Queue

Provides a hybrid queuing implementation for MCE supporting normal queues and priority queues from a single module. MCE::Queue exchanges data via the core engine to enable queueing to work for both children (spawned from fork) and threads.

MCE::Relay

Enables workers to receive and pass on information orderly with zero involvement by the manager process while running.

MCE MODELS ^

The models take Many-Core Engine to a new level for ease of use. Two options (chunk_size and max_workers) are configured automatically as well as spawning and shutdown.

MCE::Loop

Provides a parallel loop utilizing MCE for building creative loops.

MCE::Flow

A parallel flow model for building creative applications. This makes use of user_tasks in MCE. The author has full control when utilizing this model. MCE::Flow is similar to MCE::Loop, but allows for multiple code blocks to run in parallel with a slight change to syntax.

MCE::Grep

Provides a parallel grep implementation similar to the native grep function.

MCE::Map

Provides a parallel map model similar to the native map function.

MCE::Step

Provides a parallel step implementation utilizing MCE::Queue between user tasks. MCE::Step is a spin off from MCE::Flow with a touch of MCE::Stream. This model, introduced in 1.506, allows one to pass data from one sub-task into the next transparently.

MCE::Stream

Provides an efficient parallel implementation for chaining multiple maps and greps together through user_tasks and MCE::Queue. Like with MCE::Flow, MCE::Stream can run multiple code blocks in parallel with a slight change to syntax from MCE::Map and MCE::Grep.

MISCELLANEOUS ^

Miscellaneous additions included with the distribution.

MCE::Candy

Provides a collection of sugar methods and output iterators for preserving output order.

MCE::Examples

A placeholder for the examples included with the distribution. Describes various demonstrations for MCE including a Monte Carlo simulation.

MCE::Subs

Exports functions mapped directly to MCE methods; e.g. mce_wid. The module allows 3 options; :manager, :worker, and :getter.

REQUIREMENTS ^

Perl 5.8.0 or later. PDL::IO::Storable is required in scripts running PDL.

SOURCE ^

The source is hosted at http://code.google.com/p/many-core-engine-perl/

AUTHOR ^

Mario E. Roy, <marioeroy AT gmail DOT com>

COPYRIGHT AND LICENSE ^

Copyright (C) 2012-2015 by Mario E. Roy

MCE is released under the same license as Perl.

See http://dev.perl.org/licenses/ for more information.

syntax highlighting: