View on
MetaCPAN is shutting down
For details read Perl NOC. After June 25th this page will redirect to
BBC (British Broadcasting Corporation) > Sub-Slice > Sub::Slice::Backend


Annotate this POD


New  1
Open  0
View/Report Bugs


Sub::Slice::Backend - API for Sub::Slice backends


        use Sub::Slice;
        use Sub::Slice::Backend::MyBackend;
        our %Options = ( ...options for your backend... );
        sub create_token {
                my $job = new Sub::Slice (
                        backend => 'MyBackend',
                        storage_options => \%Options,
                return $job->token;
        sub do_work {
                my $job = new Sub::Slice (
                        backend => 'MyBackend',
                        storage_options => \%Options,
                        token => shift()


This is the API which storage backends for Sub::Slice must implement. Sub::Slice comes with a default Filesystem backend using Storable to serialise data. If you'd rather store the Sub::Slice data somewhere else or in another format, you can create an adaptor class satisfying this API to plug Sub::Slice into your storage system.

Module naming conventions

If a string matching \w+ is passed to Sub::Slice for the backend, the module is assumed to be in the Sub::Slice::Backend namespace (such as the example in the SYNOPSIS). If you have a module in a namespace other than the top-level one, e.g. MySystem::StorageModule, that exposes this API, Sub::Slice will happily use this as a backend module without you needing to create a wrapper in the Sub::Slice::Backend namespace.

        my $job = new Sub::Slice (
                backend => 'MySystem::StorageModule',
                storage_options => \%Options,


$backend = new Sub::Slice::Backend::YOURCLASS(\%options)

Contructor takes a hash of options

$id = $backend->new_id()

Generates a new ID.

$job = $backend->load_job($id)

Loads an existing job from persistent storage


Saves a job to persistent storage


Removes a job from persistent storage

$backend->store($job, $key, $data)

Store data for job

$data = $backend->fetch($job, $key)

Fetch data for job

$backend->store_blob($job, $key, $data)

Store BLOB data for job

$data = $backend->fetch_blob($job, $key)

Fetch BLOB data for job

$count = $backend->cleanup($age)

Deletes any leftover data which has been unmodified for at least $age days (default 1 day). $count will equal the number of files or items deleted, or may be undef if cleanup wasn't able to scan for leftover data (eg. because a directory didn't exist). Should die if it doesn't manage to clean up (eg. because of a permissions problem).

Normally, tasks should clean up after themselves, so unless you are experiencing errors, cleanup will have nothing to do.

Strategies for issuing IDs ^

The storage API requires that you can issue an ID for a new record before storing data into it. In an Oracle database you normally use a sequence to generate a unique ID in advance of storing a record so this isn't a problem. In MySQL, if you are planning on using an AUTO_INCREMENT column to generate ids, you're probably best off inserting an empty row. Alternatively in most database engines you can simulate an Oracle sequence using a counter table (see the example in the MySQL manual for a neat example using LAST_INSERT_ID). If your storage engine doesn't have a unique ID mechanism, a couple of strategies are open to you: you can use a perl GUID generator (such as Data::UUID), or you can roll your own sequence generator as in the example below.

Worked Example ^

Here's a sample implementation. It's not a particularly good one in that it doesn't handle concurrent processes writing to the DBM file, but it does illustrate the steps required to write a backend.

        package Sub::Slice::Backend::MLDBM;
        use GDBM_File;
        use Storable();
        use MLDBM qw(GDBM_File Storable);

The constructor takes a hashref of storage options, which you're free to define the meaning of. These are what the caller will pass in as the storage_options to Sub::Slice.

        sub new {
                my ($class, $options) = @_;
                my $dbm = $options->{DBM} or die("You must supply a DBM");
                my %db;
                tie(%db, 'MLDBM', $dbm, GDBM_WRCREAT, 0666) or die("unable to tie to $dbm");
                my $self = {$db => \%db};
                return bless($self, $class);

        sub DESTROY {
                my $self = shift;
                untie %{$self->{db}} if tied %{$self->{db}};

The new_id routine should create a unique ID which can be used to store the job against. Here we use a sentinel key to store a sequence counter against:

        sub new_id {
                my ($self) = @_;
                my $id = ++$self->{db}->{__COUNT__};
                return $id;

The delete_job routine takes an ID rather than a job:

        sub delete_job {
                my ($self, $id) = @_;
                delete $self->{db}->{$id};

This allows it to be used in cleaner processes without loading the job first. The load_job routine should return the job given the ID:

        sub load_job {
                my ($self, $id) = @_;
                return $self->{db}->{$id};

The save_job should persist the job (against the ID):

        sub save_job {
                my ($self, $job) = @_;
                $self->{db}->{$job->{id}} = $job;

Store and fetch methods are passed the job object so they can hang data on it, if the backend desires:

        sub store {
                my ($self, $job, $key, $value) = @_;
                $job->{data}{$key} = $value;
        sub fetch {
                my ($self, $job, $key) = @_;
                return $job->{data}{$key};

If there is no optimisation you want to perform for BLOB data, the following implementation is legitimate:

        sub store_blob {
        sub fetch_blob {


$Revision: 1.8 $ on $Date: 2004/12/17 14:41:21 $ by $Author: johna $


John Alden <cpan _at_ bbc _dot_ co _dot_ uk>


(c) BBC 2004. This program is free software; you can redistribute it and/or modify it under the GNU GPL.

See the file COPYING in this distribution, or

syntax highlighting: