Herald van der Breggen > Queue-Q-0.15 > Queue::Q::ReliableFIFO

Download:
Queue-Q-0.15.tar.gz

Dependencies

Annotate this POD

CPAN RT

New  1
Open  0
View/Report Bugs
Source   Latest Release: Queue-Q-0.24

NAME ^

Queue::Q::ReliableFIFO - FIFO queue keeping track of claimed items

SYNOPSIS ^

  use Queue::Q::ReliableFIFO::Redis; # or ::Perl or ...
  my $q = ... create object of chosen ReliableFIFO implementation...
  
  # producer:
  $q->enqueue_item([qw(my data structure)]); # rinse repeat...
  
  # consumer:
  my $item = $q->claim_item;
  # work with data...
  $q->mark_item_as_done($item);
  
  # Implementation dependent: somewhere in a recovery cron job
  # - Fetch claimed items older than $n minutes/seconds
  # - Requeue or log&drop those timed-out items

DESCRIPTION ^

Abstract interface class for a FIFO queue that keeps track of all in-flight ("claimed") items. Implementations are required to provide strict ordering and adhere to the run-time complexities listed below (or better).

The general workflow with Queue::Q::ReliableFIFO based queue is:

To recover from failed consumers, one may apply in any one of many application specific recovery strategies such as periodically re-enqueuing all claimed items that are older than a threshold or possible simply clearing them out and logging the fact.

Since the actual recovery strategy is application-dependent and the support by the queue implementation may vary, there's no API for this in this abstract base class. That may change in a future release.

METHODS ^

enqueue_item

Given a data structure, that data structure is added to the queue. Items enqueued with enqueue_item in order must be returned by claim_item in the same order.

Complexity: O(1)

enqueue_items

Given a number data structures, enqueues them in order. This is conceptually the same as calling enqueue_item multiple times, but may help save on network round-trips.

Complexity: O(n) where n is the number of items to enqueue.

claim_item

Returns the oldest item in the queue. Returns undef if there is none left.

Complexity: O(1)

Implementations may (but should not) deviate from the strict O(1) complexity for the number of claimed items at any given time. That is acceptable as high as O(log(n)). This is likely acceptable because the number of items being worked on is likely not to be extremely large. Implementations that make use of this relaxed requirement must document that clearly. The number of queued items must not affect the complexity, however.

claim_items

As enqueue_items is to enqueue_item, claim_items is to claim_item. Takes one optional parameter (defaults to 1): The number of items to fetch and return:

  my @items = $q->claim_items(20); # returns a batch of 20 items

If there are less than the desired number of items to be claimed, returns a correspondingly shorter list.

Complexity: O(n) where n is the number of items claimed.

See the documentation of claim_item for a potential relaxation on the complexity bounds with respect to the number of in-flight, claimed items.

mark_item_as_done

Given a item that was previously claimed from this queue, it is removed from the claimed-items tracking and thus removed from the queue altogether.

Complexity: Generally O(1), but O(log(n)) under relaxed requirements (see above).

mark_items_as_done

Given any number of items that were previously claimed from this queue, they are removed from the claimed-items tracking and thus removed from the queue altogether.

Complexity: Generally O(n), but O(n*log(n)) under relaxed requirements (see above) with n understood to be the number of items to mark as done.

queue_length

Returns the number of items available in the queue.

Complexity: O(1)

flush_queue

Removes all content in the queue.

Complexity: O(n) where n is the number of items in the queue.

AUTHOR ^

Herald van der Breggen, <herald.vanderbreggen@booking.com>

COPYRIGHT AND LICENSE ^

Copyright (C) 2012 by Herald van der Breggen

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.1 or, at your option, any later version of Perl 5 you may have available.

syntax highlighting: