Steven Lembark > Parallel-Queue-v3.0.4 > Parallel::Queue



Annotate this POD (1)

Related Modules



Open  0
View/Report Bugs
Module Version: v3.0.4   Source   Latest Release: Parallel-Queue-3.2


Parallel::Queue - fork list of subref's N-way parallel


    # example queue:
    # only squish files larger than 8KB in size.  figure
    # that the system can handle four copies of squish
    # running at the same time without them interfering
    # with one another.

    my @queue = map { -s > 8192 ? sub{ squish $_ } : () } @filz;

    # simplest case: use the module and pass in 
    # the count and list of coderefs.

    use Parallel::Queue;

    my @remaining = runqueue 4, @queue;

    die "Incomplete jobs" if @remaining;

    # if the next item on the queue can "next_job" 
    # then the return value will be dispatched 
    # until the call to next_job returns false.
    # $iter can be an object or class.
    # note that the job is dispatched *after* a
    # fork, which means that it should extract
    # any necessary information from control
    # variables prior to generating the next job.

    my $iter    = Foo->new( @job_parmz );
    runqueue 4 => $iter;

    package Foo;

    my @valz    = ( ... );

    sub next_job
        @valz or return;

        my $next = shift @valz;

        sub { frobnicate $next }

    # export allows changing the exported sub name.
    # "export=" allows not exporting it (which then
    # requires calling Parallel::Queue::runqueue ...

    use Parallel::Queue qw( export=handle_queue );

    my @remaining = handle_queue 4, @queue;

    # "fork" is the normal state.
    # "nofork" or a zero count avoid forking.
    # detecting the perl debugger (via $^P) will
    # defaults "fork" to false ("nofork" mode).
    # forking in the debugger can be turned on
    # with an explicit fork (which includs a 
    # warning if $DB::DEBUG_TTY is not set).

    use Parallel::Queue;                # fork by default unless $^P.

    use Parallel::Queue qw( fork   );   # forks, even in the debugger.
    use Parallel::Queue qw( nofork );   # don't attempt forks

    # "debug" turns on nofork and verbose.
    # these produce identical results.

    use Parallel::Queue qw( debug );        
    use Parallel::Queue qw( nofork verbose );

    # finish forces execution to continue even if 
    # there is an error in one job. this will finish
    # the cleanups even if one of them fails.

    use Parallel::Queue qw( finish );

    my @cleanupz    = ... ;

    runqueue $nway, @cleanupz;

    # "configure" is a more descriptive alias for the
    # import sub that defaults to 'noexport':

    Parallel::Queue->configure( debug=0 finish=1 );


Arguments to use (or configure).

The finish, debug, and verbose arguments default to true. This means that turning them on does not require an equals sign and value. Turning them off requries an equal sign and zero.

The export option defaults to false: using it without a value avoids exporting the runqueue subroune into the caller.

finish =item finish=0

This causes the queue to finsih even if there are non-zero exits on the way. Exits will be logged but will not stop the queue.

export=my_name =item export=

By default Parallel::Queue exports "runqueue", this can be changed with the "export" arguments. In this case call it "my_name" and use it to run the queue with two parallel processes:

    use Parallel::Queue qw( export=run_these );

    my @queue       = ...;

    my @un_executed = run_these 2, @queue;

The name can be any valid Perl sub name.

Using an empty name avoids exporting anything and requires using the fully qualified subname (Parallel::Query::runqueue) or a subref (via Parallel::Queue->can( 'runqueue' )) to run the queue.


This outputs a line with the process id each time a job is dispatched or reaped.


Turned on by default if the perl debugger is in use (via $^P), this avoids forking and simply dispatches the jobs one by one. This helps debug dispatched jobs where handling forks in the Perl debugger can be problematic.

Debug can be turned off via debug=0 with the use or confgure. If this is turned off with the debugger running then be prepared to supply the tty's (see also Debugging forks below).


Non-numeric count arguments.

The runqueue sub uses Scalar::Util::looks_like_number validate the count. This may cause problems for objects which don't look like numbers.


Debugging forks.



This code is released under the same terms as Perl-5.12 or any later version of Perl.


Steven Lembark <>

syntax highlighting: