The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
ElasticSearchX-Sequence-0.01
++
/ ElasticSearchX::Sequence

NAME

ElasticSearchX::Sequence - Fast integer ID sequences with ElasticSearch

VERSION

version 0.01

SYNOPSIS

    use ElasticSearch();
    use ElasticSearchX::Sequence();

    my $es  = ElasticSearch->new();
    my $seq = ElasticSearchX::Sequence->new( es => $es );

    $seq->bootstrap();

    my $it  = $seq->sequence('mail_id');

    my $mail_id = $it->next;

DESCRIPTION

ElasticSearchX::Sequence gives you a sequence of auto-incrementing integers (eg to use as IDs) that are guaranteed to be unique across your application.

It is similar in spirit to DBIx::Sequence, but uses ElasticSearch as a backend.

MOTIVATION

ElasticSearch already has built in unique IDs, but they look like this: KpSb_Jd_R56dH5Qx6TtxVA.

If you are migrating from an RDBM where you are using (eg) an auto-increment column to give you unique IDs, your application may depend on these IDs being integers. Or you may just prefer integer IDs.

Either way, this module makes it easy to get these unique auto-incrementing IDs without needing an RDBM to provide them.

And it is fast! Given the performance, if you are already using ElasticSearch, you may want to move your ticket servers from your database to ElasticSearch instead.

PERFORMANCE

This module is blazing fast, especially when ElasticSearch uses the ElasticSearch::Transport::Curl backend.

You can try out the benchmark yourself, in the benchmark folder in this distribution.

The script compares:

The results I get when running this on my laptop are:

                   Rate es_curl_10  db_ticket    es_tiny    es_curl
    es_curl_10  38760/s         --       -48%       -55%       -72%
    db_ticket   74627/s        93%         --       -13%       -47%
    es_tiny     85470/s       121%        15%         --       -39%
    es_curl    140845/s       263%        89%        65%         --

Plus, with ElasticSearch, you get distributed and high-availability thrown in for free.

METHODS

new()

    my $seq = ElasticSearchX::Sequence->new(
        es      => $es,         # ElasticSearch instance, required
        index   => 'index',     # defaults to 'sequence',
        type    => 'type',      # defaults to 'sequence',
    );

new() returns a new instance of ElasticSearchX::Sequence. By default, your sequences will be stored in index sequence, type sequence, but you can change those values to whatever suits your application.

By default, the index is optimised for serving sequences, and has different settings than those you would typically use in your main index, so rather than storing your sequences in the main index for your application(s), you may prefer to store all of your sequences in the single index sequence.

The type (default sequence) could be used to separate sequences for different applications. For instance, you could store the sequences for your personal blog in type personal and for your work blog in type work.

See "bootstrap()" for how to initiate your index/type.

sequence()

    my $it = $seq->sequence('mail_id');
    my $it = $seq->sequence( name => 'mail_id', size => 100 );

The sequence() method returns a new sequence iterator identified by the name.

New IDs/values are generated in blocks of size (default 100), as this is much faster than requesting them individually.

This does mean that, if you have several instances of the iterator mail_id, then the next ID won't always be the highest number available. For instance:

    $i_1 = $seq->('mail_id');
    $i_2 = $seq->('mail_id');

    say $i_1->next;
    say $i_2->next;
    say $i_1->next;
    # 1
    # 101
    # 2

See also ElasticSearchX::Sequence::Iterator.

bootstrap()

    $seq->bootstrap( %settings );

This method will create the index, if it doesn't already exist, and will setup the type. This can be called even if the index and type have already been setup. It won't fail unless the type already exists and has a different mapping / definition.

By default, the index is setup with the following %settings:

    (
        number_of_shards     => 1,
        auto_expand_replicas => '0-all',
    )

In other words, it will have only a single primary shard (instead of the ElasticSearch default of 5), and a replica of that shard on every ElasticSearch node in your cluster.

If you pass in any %settings then the defaults will not be used at all.

See Index Settings for more.

delete_index()

    $seq->delete_index()

Deletes the index associated with the sequence. You will lose your data!

delete_type()

    $seq->delete_type()

Deletes the type associated with the sequence. You will lose your data!

index()

    $index = $seq->index

Read-only getter for the index value

type()

    $type = $seq->type

Read-only getter for the type value

es()

    $es = $seq->es

Read-only getter for the ElasticSearch instance.

SEE ALSO

ElasticSearch, http://www.elasticsearch.org

BUGS

If you have any suggestions for improvements, or find any bugs, please report them to https://github.com/clintongormley/ElasticSearchX-Sequence/issues. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

AUTHOR

Clinton Gormley <drtech@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2011 by Clinton Gormley.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.