Joshua Nathaniel Pritikin > ObjStore-1.59 > ObjStore

Download:
ObjStore-1.59.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Module Version: 1.59   Source  

NAME ^

ObjStore - Perl Extension For ObjectStore OODBMS

SYNOPSIS ^

Like perl? Sick of SQL? Ready to try something new? This extension implements something like a relational database management system, except that s/SQL/Perl/ig.

DESCRIPTION ^

ObjectStore is the market leader in object-oriented databases http://www.odi.com (NASDAQ: ODIS). They use a unique Cache-Forward Architecture to make persistent data available in literally the most efficient manner possible.

Prior to this joining of forces

Now there is an easy way to build database applications (especially if you are concerned about preserving your ideals of encapsulation).

WHAT IS THIS GREEN SLIME?? ^

Maybe you're feeling like you're in unfamiliar territory? You see green slime oozing from the ceiling. Something doesn't smell right and a shiver runs through hair on the back of your neck. I hope you're a quick learner! Many people haven't made it.

Learning new stuff is hard work. While this may be true, isn't it easier to get ahead than to be left behind? It's a competitive, preditory world. Which is really easier?

QUICK START ^

The first step is to make sure that ObjStore is loaded.

  use ObjStore;  #load ObjStore

Persistent perl is just like normal perl, except that you can easily create data that doesn't go away when your program exits (or core dumps :-). This more permenant data lives in databases stored in files or raw disk partitions. And databases are comprised of...

Segments

Segments dynamically resize from very small (less than 64k) to very large (~2GB). Segments can be used to cluster data and can be a unit of locking or caching. You can find the segment for any persistent object with the segment_of method.

  $db->segment_of;            # default segment
  $any_object->segment_of;    # the segment of $any_object

How to create/open the database?

  use ObjStore::Config ('$TMP_DBDIR');
  my $db = ObjStore::open("$TMP_DBDIR/myjunk", 'update', 0666);

This creates or opens the database 'myjunk' in the $TMP_DBDIR directory. ($TMP_DBDIR could be any directory on the server's local filesystem.) The database is opened in 'update' mode (also see 'mvcc'). If the database doesn't exist then it is created with access 0666. The file mode 0666 is specified in the same format as is understood by chmod. If osserver is running as root then the file mode will protect database exactly same way that it protects files.

How to add stuff to the database?

  begin 'update', sub {
    $db->root('top', sub{ { tag => "This is the top" } });
  };
  die if $@;

Here we add the root 'top' to the database if it doesn't already exist. 'top' is a hash reference that contains one key-value pair (tag => "This is the top"). In general, you should not need more than one root per database.

What are transactions?

In order to read or write persistent data, you must be inside a transaction. Transactions can be read-only ('read') or both read and write ('update'). Transactions guarentee a consistent view within a database but they do not guarentee a consistent view between databases. Locking is accomplished per page or at a higher level of granularity. Deadlocks are resolved by an application determined transaction priority. In perl, transactions work similarly to eval {}.

  begin 'read', sub { ... };   die if $@;    # read transaction
  begin 'update', sub { ... }; die if $@;    # update transaction
  begin sub { ... };           die if $@;    # nested transaction

To abort a transaction, simply use 'die'. The 'die if $@' (above) is used to propagate errors to the top-level eval context. If you forget to check $@ then your scripts may fail for mysterious reasons without reporting any kind of diagnostic.

How to read stuff from the database?

Reading data is very similar to writing data.

  begin 'read', sub {
    my $top = $db->root('top');
    warn $top->{tag};  # This is the top
  };
  die if $@;

If you attempt to update persistent data in a read transactions then an exception will be thrown.

Transaction Scope

Persistent objects are scoped to their transaction. This means that objects become unreachable once their transaction commits or aborts. For example,

  my $top;
  begin 'read', sub { $top = $db->root('top') }; die if $@;
  warn $top->{tag};        # EXCEPTION THROWN

If one needs to keep a reference to an object between transactions then this can be accomplished with ObjectStore references.

  my $top;
  begin 'read', sub {
    $top = $db->root('top')->new_ref
  };
  die if $@;
  warn $top->focus->{tag};        # EXCEPTION THROWN

  begin 'read', sub {
    warn $top->focus->{tag};      # OK
  };
  die if $@;

A brief note about reference counting

Data stored in the database is reference counted. This means that you (usually) don't have to worry about explicitly deallocating usused memory. However, be aware that transient reference do not count towards a persistent reference count. For example,

  $phash->{obj} = {};              # create a fresh persistent hash
  my $obj = delete $phash->{obj};  # remove from the database
  warn $obj->{attr};               # SEGV (or worse)

If you suspect a refcnt problem but are having difficulty tracking it down then you may compile with OSP_SAFE_BRIDGE=1 (see ObjStore::MakeMaker). Fortunately however, errors of this nature are usually obvious and easy to fix.

Tools

DATA TYPES ^

Hashes

The following code snippet creates a persistent hash reference with an expected cardinality of ten elements.

    my $h7 = ObjStore::HV->new($store, 10);

A simple array representation is used for low cardinalities. Arrays do not scale well, but they do afford a pleasingly compact memory footprint. ObjectStore's os_Dictionary is transparently used for large cardinalities [MAYCHANGE].

Persistent data structures can be built with the normal perl construction:

    $h7->{foo} = { 'fwaz'=> { 1=>'blort', 'snorf'=>3 }, b=>'ouph' };

Or the equally effective, but unbearibly tedious:

    my $h1 = $dict->{foo} ||= ObjStore::HV->new($dict);
    my $h2 = $h1->{fwaz} ||= ObjStore::HV->new($h1);
    $h2->{1}='blort';
    $h2->{snorf}=3;
    $h1->{b}='ouph';

Perl saves us again!

Arrays

The following code snippet creates a persistent array reference with an expected cardinality of ten elements.

    my $a7 = ObjStore::AV->new($store, 10);

Once you have a persistent array, everything else is mostly the same as any other type of array. All the usual array operations will just work.

However, complete array support requires at least perl 5.004_57. If you need to upgrade, consider doing so.

References

You can generate a reference to any persistent object with the method new_ref($segment, $type). Since reference do not affect reference counts, they are a safe way to refer across databases. They can also be allocated transiently. Transient references are useful because they remain valid across transaction boundries.

  $r->open($how);              # attempts to open the focus' database
  $yes = $r->deleted;          # is the focus deleted?
  $f = $r->focus;              # returns the focus of the ref
  $str = $r->dump;             # the ref as a string

Be aware that references can return garbage if they point to data in a database that is not open. You will need explicitly open them if in doubt (see ObjStore::Ref::POSH_ENTER). For example,

  $r = $o->new_ref('transient');

  $r = $o->new_ref();    # same as above

This creates an os_reference. Care must to taken that these hard reference do not point to objects that have already been deleted. A SEGV (if you're lucky) or garbled data can result.

Fortunately, it is always safe to use hard references in the case that they are used to merely to avoid circular references within a single database. For example:

  my $o = ObjStore::HV->new($db);
  $$o{SELF} = $o->new_ref($o,'hard');

The reference in hash slot 'SELF' can be used by any part of the downwardly nested data structure to point back to the top-level.

A database itself is somewhat similar in the sense that every object has a database_of method. You might say that every member of a database has an implicit reference to the top-level of the database.

Indices

SQL has this fantastic facility called indices. ObjStore does too!

  my $nx = ObjStore::Index->new($near);
  $nx->configure(unique => 1, path=>"name");

  $nx->add({name=> 'Square'}, {name=> 'Round'}, {name=> 'Triangular'});

  my $c = $nx->new_cursor;
  $c->seek('T');
  $c->step(1);
  warn $c->at()->{name}; # Triangular

Be aware that no index representations are supplied by default. You cannot create indices without another extension (such as ObjStore-REP-FatTree).

Indices should not be used unless other data structures prove inadequate. Some indices cannot be allocated transiently. Simple hashes and arrays are less trouble in most cases. Indices are best suited to large data-sets that must be indexed in more than a few ways. If you can always afford to iterate through every record then indices probably aren't worth the effort.

Index Cursors

Index cursors are a lot more powerful than hash or array cursors. The methods available are:

  $c->focus();
  $c->moveto($record_no);
  $c->step($delta);
  $c->seek(@keys);
  my $pos = $c->pos();            
  my @keys = $c->keys();
  my $v = $c->at();
  my $v = $c->each($delta);

Where the following invariants hold:

  $c->moveto($c->pos() + $delta) is $c->step($delta)
  $c->each($delta) is { $c->step($delta); $c->at(); }

Be aware that index cursors may only be used by one thread at a time. Thus, it is usually not worthwhile to store pre-created cursors persistently. Create transient ones as needed.

Index Representations

Certain representations for indices are very sensitive to the proper order of the records that they contain (trees in particular). To eliminate the possibility of indices diverging with indexed data, keys are marked read-only when records are added to the index. This insures that indices will not be disordered by ad-hoc updates to records' keys. (Just imagine if people were constantly changing the meanings of words as you read this sentance.)

For example:

  my $category = { name => 'Bath Toy' };
  my $row = { name => 'Rubber Ducky', category => $category };

  $index->configure(path => 'category/name, name');
  $index->add($row);

  $row->{category}{name} = 'Beach Toy';  # READONLY
  $row->{name} = "Rubber Doggie";        # READONLY
  $row->{owner} = $bob;                  # ok

The first key is the category's name. The second key is the row's name.

Another scheme for keeping indices up-to-date is to use os_backptr. This scheme is not supported because it has considerable memory overhead (12-bytes per record!) and provides little benefit beyond the read-only scheme.

The exact constraints of indexing are dependent on the index representation and the record representation. Different representations can allow a greater or lesser degree of flexibility. If you need something that is not supported by an existing extension, it is likely that the behavior you need can be implemented with new representations. (Virtual sunglasses not included. :-)

DATABASE DESIGN ^

The best design is to be flexible!

The best design is to be flexible!

The best design is to be flexible!

Wait! No Schema?! How Can This Scale? Wont I get lost in a morass of disorganization and change-control meetings?

How can a relational database scale?! When you write down a central schema you are violating the principle of encapsulation. This is dumb. While relational databases might have been a revelation when they were conceived, why is it still standard practice to create artificial dependencies and breaches of abstraction in database design? It is entirely avoidable!

The Theory of Lazy Evolution

When you practice lazy evolution, you avoid changing data layouts in favor of making queries smarter.

This is the correct trade because data usually out-bulks code. Furthermore, since there isn't a artificial split between the implementation language and a goofy, unempowered database language, schema evolution is reduced to the same problem as source code compatibility. (A database is just a rich API.)

Now, I'm not saying that data layouts never have to change. When thinking about data the first thing to consider is how to organize everything to ease one's ability to evolve.

First of all, it is entirely unnecessary to store all data in a single database. Multiple databases interoperate and can be used simultaniously. For example, you can have a separate database for each day or each month. By starting fresh with a new database it is easier to make structural changes while perserving backward compatibility. In constrast to relational databases, the scope of the evolution problem can be easily delineated.

Within a database, the way you split up your database generally depends on three things:

A few additional considerations can be found in the ObjectStore documentation, but that's about it. Don't over-complicate things. This isn't a relational database, remember? The power and simplicity is hard to describe because there's just not much to it. (Just the absolute minimum to get the job done. :-)

RDBMS Emulation

Un-structured perl databases are probably under-constrained for most applications. Fortunately, RDBMS style tables are adapted, adopted, and included within this package. While they are a little different from traditional tables, I am confident that with a few doses of Prozac relational developers will feel right at home. See ObjStore::Table3 and ObjStore::Index.

THE ADVANCED CHAPTER ^

Performance Check List

The word tuning implies too high a brain-level requirement. Getting performance out of ObjectStore is not rocket science. On the other hand, you shouldn't think that perl will ever go as fast as optimized C++. That's what DLL schemas are for. For an example, see ObjStore::REP::HashRecord.

Transactions Redux

Stargate Mechanics

Here is how to create hashes and arrays pre-sized to exactly the right number of slots:

  ObjStore::HV->new($near, { key => 'value' });  # 1 slot
  ObjStore::AV->new($near, [1..3]);              # 3 slots

Or you can interface the stargate directly:

  my $persistent_junk = ObjStore::translate($near, [1,2,3,{fat=>'dog'}]);

If you want to design your own stargate, you may inspect the default stargate in ObjStore.pm for inspiration. (Not recommended. ;-)

How Can I Rescue Persistent Objects From Oblivion?

All data stored in ObjectStore is reference counted. This is a fantastically efficient way to manage memory (for most applications). It has very good locality and low overhead. However, as soon as an object's refcnt reaches zero, it is permenantly deleted from the database. You only have one chance to save the object. The NOREFS method is invoked just prior to deletion. You must hook it back into the database or kiss the object goodbye.

Be aware that the DESTROY method is still invoked every time an object becomes unreachable from the current scope. However, contrary to transient objects this method does not preview persistent object destruction. (Hacking DESTROY such that it is used instead of NOREFS is desirable but would require changes to the perl code-base. This change is under consideration. Let me know if you care. XXX)

Also see ObjStore::Mortician!

posh

posh is your command-line window into databases.

posh obediently tries to treat your data in an application specific manner. Customize by providing your own implementation of these methods:

There are lots of good examples throughout the standard ObjStore:: libraries. Also see ObjStore::Peeker.

Arrays-as-Hashes

  use base 'ObjStore::AVHV';
  use fields qw(f1 f2 f3);

Fantastically efficient records hash records. See ObjStore::AVHV and fields.

Autoloading

As you use a database, ObjStore tries to require each class that doesn't seem to be loaded. To disable class autoloading behavior, call this function before you open any databases:

  ObjStore::disable_class_auto_loading();

This mechanism is orthogonal to perl's AUTOLOAD mechanism for autoloading functions.

Where Is The Hard Part?

I don't know. Humbly I am working to trying to find out.

DIRECTION ^

How Does CORBA Fit In?

CORBA standardizes remote method invokation (RMI). ObjectStore greatly reduces the need for remote method invokation and also provides a simple but effective RMI mechanism (see ObjStore::notify and ObjStore::ServerDB). The two technologies address different problems but here is a rough comparison:

 GENERALLY            CORBA                     ObjectStore
 -------------------- ------------------------- -------------------------
 flow                 you follow the data       the data comes to you
 network              central assumption        what network?
 flexibility          brittle                   up to you
 object-oriented      yes                       up to you
 data copying         2-4 times                 0-2 times
 binary portability   yes                       somewhat
 reference counted    no (does not store data)  yes (at least for perl)
 multi-vendor         yes                       not yet

 MESSAGING            CORBA                     ObjectStore
--------------------- ------------------------- ------------------------
 reliable             yes?                      no
 a/syncronous         both?                     async only

Why Is Perl a Better Fit For Databases Than SQL, C++, or Java?

  struct CXX_or_Java_style {
        char *name;
        char *title;
        double size;
  };

When you write a structure declaration in C++ or Java you are declaring field-names, field-types, and field-order. Programs almost always require a re-compile to change such rigid declarations. This is fine for small applications but becomes cumbersome quickly. It is too hard to change (brittle). An SQL-style language is needed. When you create a table in SQL you are declaring only field-names and field-types.

  create table SQL_style
  (name varchar(80),
   title varchar(80),
   size double)

This is more flexible, but SQL gives you far less expressive power than C++ or Java. Applications end up being written in C++ or Java while their data is stored with SQL. Managing the syncronization between the two languages creates enormous extra complexity. So much so that there are lots of software companies that exist solely to address this headache. (You'd think they'd try to cure the problem directly instead of addressing the symptom!) Perl is better because it transparently spans all the requirements in a single language.

  my $h1 = { name => undef, title => undef, size => 'perl' };

Only the field-names are specified. It should be clear that this declaration is even more flexible than SQL. The field-types are left dynamic. Actually, even the field-names are not fixed. The flexibility potential is very great. It might even be too flexible!

Fortunately, perl anticipated this. Perl is flexible about how flexible it is. Therefore, if you need rigidity and the speed that comes with it, you have recourse. Specific objects or subsystems can be implemented directly in C++. It's win-win (at least!).

Why Is Perl Easier Than All Other Programming Languages?

I have no idea!

Summary (LONG)

Summary (SHORT)

Perl can store data

without violating the encapsulation principle or obstructing general ease of use.

EXPORTS ^

bless and begin by default. Most other static methods can also be exported.

ENVIRONMENT VARIABLES ^

AUTHOR ^

Copyright © 1997-2000 Joshua Nathaniel Pritikin. All rights reserved.

This package is free software and is provided "as is" without express or implied warranty. It may be used, redistributed and/or modified under the terms of the Perl Artistic License (see http://www.perl.com/perl/misc/Artistic.html)

The Perl / ObjectStore extension is available via any CPAN mirror site. See http://www.perl.com/CPAN/authors/id/JPRIT/

Portions of the collection code were derived from splash, Jim Morris's delightful C++ library ftp://ftp.wolfman.com/users/morris/public/splash

SEE ALSO ^

ObjStore::Reference, ObjStore::Table3, examples in the t/ directory, ObjStore::Internals, Event, PerlQt, and The SQL Reference Manual (just kidding :-)

syntax highlighting: