Robert Barta > Parallel-MapReduce > Parallel::MapReduce::Worker

Download:
Parallel-MapReduce-0.09.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Module Version: 0.03   Source  

NAME ^

Parallel::MapReduce::Worker - MapReduce, local worker

SYNOPSIS ^

  use Parallel::MapReduce::Worker;
  my $w = new Parallel::MapReduce::Worker;

  my @chunks = chunk_n_store ($memd, $A, $job, 1000);
  my $cs = $w->map (\@chunks, 'slice1:', SERVERS, $job);
  ...
  my $ks = $w->reduce (\@cs, SERVERS, $job);

DESCRIPTION ^

This class implements a local, sequential worker. You will only know about it if you want to subclass it to implement your own worker.

INTERFACE ^

Constructor

Nothing important to be said.

Destructor

$w->shutdown

While there is no DESTROY method (for technical reasons), there is a shutdown method. It is supposed to terminate any background processes a worker might have started.

For a single-thread, local version nothing will be done, but individual subclasses might have to do substantial work to tear-down network connections, remote servers, etc.

Methods

map

$keys = $w->map ($chunks, $slice, $servers, $job)

The chunks are keys into the memcached servers. They allow to reconstruct the hash slice to be worked on. The slice is a simple id for that hash slice. The list reference to the memcached servers is obviously also necessary and the job id is an identifier for the current MR computation.

reduce

$chunks = $w->reduce ($keys, $servers, $job)

The keys are the keys into the intermediate hash within the memcached servers. The list reference to the memcached servers is obviously also necessary and the job id is an identifier for the current MR computation.

SEE ALSO ^

Parallel::MapReduce

COPYRIGHT AND LICENSE ^

Copyright 200[8] by Robert Barta, <drrho@cpan.org>

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

syntax highlighting: