NAME

DynGig::Util::MapReduce - A Map Reduce Job Launcher

SYNOPOSIS

 use DynGig::Util::MapReduce;

 my $batch = sub { ..  };
 my $map = sub { .. };
 my $reduce = sub { .. };

 my $job = DynGig::Util::MapReduce->new
 (
     name => 'foo',
     batch => $batch,
     map => $map,
     reduce => $reduce,
 );

 my %batch_param = ( .. );
 my %map_param = ( .. );
 my %reduce_param = ( .. );
 my %context = ( .. );

 $job->run
 (
     context => \%context,
     batch => \%batch_param,
     map => \%map_param,
     reduce => \%reduce_param,
 );

 my $result = $job->result();

DESCRIPTION

Map/Reduce is an approach that collects data in parallel then processes data in serial. A Map/Reduce job has 4 components/steps. Each component is to be defined by the user.

 Batch : divide the targets into batches.
 Map   : create threads to collect data from/for each batch in parallel.
 Sort  : aggregate collected data by status.
 Reduce: (optional) process aggregated data serially.

run( batch => HASH, map => HASH, reduce => HASH, context => HASH )

result()

Returns the result of the run as a HASH reference or undef if invoked before run.

context()

Returns the context of the run as a HASH reference or undef if invoked before run.

name()

Returns the name of the job.

SEE ALSO

threads, Thread::Queue, and YAML::XS for data serialization.

NOTE

See DynGig::Util