NAME

Job::Async::Redis - Net::Async::Redis backend for Job::Async

SYNOPSIS

 use IO::Async::Loop;
 use Job::Async;
 my $loop = IO::Async::Loop->new;
 $loop->add( my $jobman = Job::Async->new );
 my $client = $jobman->client(
     redis => { uri => 'redis://127.0.0.1', }
 );
 my $worker = $jobman->worker(
     redis => { uri => 'redis://127.0.0.1', }
 );
 Future->needs_all(
     $client->start,
 )->get;
 
 $worker->jobs->each(sub {
     $_->done('' . reverse $_->data('some_data'))
 });
 print Future->needs_all(
  $client->start,
  $worker->trigger
 )->then(sub {
  $client->submit(
   some_data => 'reverse me please'
  )->future
 })->get;

DESCRIPTION

The system can be configured to select a performance/reliability tradeoff as follows. Please note that clients and workers must be configured to use the same mode - results are undefined if you try to mix clients and workers using different modes. If it works, don't rely on it.

Operational modes

simple mode

Jobs are submitted by serialising as JSON and pushing to a Redis list as a queue.

Workers retrieve jobs from queue, and send the results via pubsub.

Multiple queues can be used for priority handling - the client can route based on the job data.

recoverable mode

As with simple mode, queues are used for communication between the clients and workers. However, these queues contain only the job ID.

Actual job data is stored in a hash key, and once the worker completes the result is also stored here.

Job completion will trigger a "publish" in Net::Redis::Async::Commands notification, allowing clients to listen for completion.

Multiple queues can be used, as with simple mode.

reliable mode

Each worker uses "brpoplpush" in Net::Async::Redis::Commands to await job IDs posted to a single queue.

Job details are stored in a hash key, as with the recoverable approach.

When a worker starts on a job, the ID is atomically moved to an in-process queue, and this is used to track whether workers are still valid.

Only one queue is allowed per worker, due to limitations of the "brpoplpush" in Net::Async::Redis::Commands implementation as described in this issue.

AUTHOR

Tom Molesworth <TEAM@cpan.org>

LICENSE

Copyright Tom Molesworth 2016-2017. Licensed under the same terms as Perl itself.