Resque - Redis-backed library for creating background jobs, placing them on multiple queues, and processing them later.
version 0.42
First you create a Resque instance where you configure the Redis backend and then you can start sending jobs to be done by workers:
use Resque; my $r = Resque->new( redis => '127.0.0.1:6379' ); $r->push( my_queue => { class => 'My::Task', args => [ 'Hello world!' ] });
Background jobs can be any perl module that implement a perform() function. The Resque::Job object is passed as the only argument to this function:
package My::Task; use strict; use 5.10.0; sub perform { my $job = shift; say $job->args->[0]; } 1;
Finally, you run your jobs by instancing a Resque::Worker and telling it to listen to one or more queues:
use Resque; my $w = Resque->new( redis => '127.0.0.1:6379' )->worker; $w->add_queue('my_queue'); $w->work;
Resque is a Redis-backed library for creating background jobs, placing them on multiple queues, and processing them later.
This library is a perl port of the original Ruby one: https://github.com/defunkt/resque My main goal doing this port is to use the same backend to be able to manage the system using ruby's resque-server webapp.
As extracted from the original docs, the main features of Resque are:
Resque workers can be distributed between multiple machines, support priorities, are resilient to memory leaks, tell you what they're doing, and expect failure.
Resque queues are persistent; support constant time, atomic push and pop (thanks to Redis); provide visibility into their contents; and store jobs as simple JSON hashes.
The Resque frontend tells you what workers are doing, what workers are not doing, what queues you're using, what's in those queues, provides general usage stats, and helps you track failures.
A lot more about Resque can be read on the original blog post: http://github.com/blog/542-introducing-resque
Redis instance for this Resque instance. Accepts a string, hash reference, Redis, Redis::Fast or any other object that behaves like those.
When a string is passed in, it will be used as the server argument of a new client object. When Redis::Fast is available this will be used, when not the pure perl Redis client will be used instead.
This is useful to run multiple queue systems with the same Redis backend.
By default 'resque' is used.
Failures handler. See Resque::Failures.
Returns a new Resque::Worker on this resque instance. It can have plugin/roles applied. See Resque::Pluggable.
my $worker = $r->worker();
Pushes a job onto a queue. Queue name should be a string and the item should be a Resque::Job object or a hashref containing:
class - The String name of the job class to run. args - Any arrayref of arguments to pass the job.
Returns redis response.
Example
$resque->push( archive => { class => 'Archive', args => [ 35, 'tar' ] } )
Pops a job off a queue. Queue name should be a string.
Returns a Resque::Job object.
my $resque_job = $r->pop( 'queue_name' );
Pops a job off an arrayref of queues prioritizing by order. Queue names should be string. It will block until a job is poped or the optional timeout in seconds.
my $resque_job = $r->blpop( [qw/ queue1 queue2 queue3/], 60 );
Returns the size of a queue. Queue name should be a string.
my $size = $r->size();
Returns a hashref with the size of an arrayref of queues. Queue names should be strings.
my $sizes = $r->size_map([qw/ queue1 queue2 queue3 /]);
This method is very fast as it will pipeline all operations.
Returns an array of jobs currently queued, or an arrayref in scalar context.
First argument is queue name and an optional secound and third are start and count values that can be used for pagination. start is the item to begin, count is how many items to return.
Passing a negative count argument will set a stop value instead of count. So, passing -1 will return full list, -2 all but last element and so on.
To get the 3rd page of a 30 item, paginatied list one would use:
my @jobs = $resque->peek('my_queue', 59, 30)
Returns an array of all known Resque queues, or an arrayref in scalar context.
my @queues = $r->queues();
Given a queue name, completely deletes the queue.
$r->remove_queue( 'my_queue' );
Given a queue name, creates an empty queue.
$r->create_queue( 'my_queue' );
Removes all matching jobs from a queue. Expects a hashref with queue name, a class name, and, optionally, args.
Returns the number of jobs destroyed.
If no args are provided, it will remove all jobs of the class provided.
That is, for these two jobs:
{ 'class' => 'UpdateGraph', 'args' => ['perl'] } { 'class' => 'UpdateGraph', 'args' => ['ruby'] }
The following call will remove both:
my $num_removed = $rescue->mass_dequeue({ queue => 'test', class => 'UpdateGraph' });
Whereas specifying args will only remove the 2nd job:
my $num_removed = $rescue->mass_dequeue({ queue => 'test', class => 'UpdateGraph', args => ['ruby'] });
Using this method without args can be potentially very slow and memory intensive, depending on the size of your queue, as it loads all jobs into an array before processing.
Build a Resque::Job object on this system for the given hashref or string(payload for object).
Resque::Job class can be extended thru roles/plugins. See Resque::Pluggable.
$r->new_job( $job_or_job_hashref );
Concatenate $self->namespace with the received array of names to build a redis key name for this resque instance.
Returns an array of all known Resque keys in Redis, or an arrayref in scalar context. Redis' KEYS operation is O(N) for the keyspace, so be careful this can be slow for big databases.
This method will delete every trace of this Resque system on the redis() backend.
$r->flush_namespace();
Does the dirty work of fetching a range of items from a Redis list.
my $items_ref = $r->list_range( $key, $stat, $count );
As in any piece of software there might be bugs around. If you found one, please report it on RT or at the github repo:
https://github.com/diegok/resque-perl
Pull requests are also very welcomed, but please include tests demostrating what you've fixed.
More tests on worker fork and signal handling.
Gearman::Client
TheSchwartz
Queue::Q4M
Diego Kuperman <diego@freekeylabs.com>
This software is copyright (c) 2021 by Diego Kuperman.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
To install Resque, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Resque
CPAN shell
perl -MCPAN -e shell install Resque
For more information on module installation, please visit the detailed CPAN module installation guide.