DESCRIPTION

This engine facilitates the inserting of hashes which describe a row of a table.

Its main feature is to remap fields in a record which refer to auto-increment columns in another table. For instance, it will correctly handle { action => 'insert', source => 'Foo', data => { id => 'x', value => 'aaaaa' } } { action => 'insert', source => 'Foo', data => { id => 'y', parent_id => 'x', value => 'bbbbb' } } { action => 'insert', source => 'Foo', data => { parent_id => 'y', value => 'ccccc' } } by translating from 'x' to the number generated for the auto_increment column <Foo.id> when inserting record 'x'. All other records which refer to <Foo.id = 'x'> will get remapped to <Foo.id = ###>. While I used strings in the example, it is just as valid to use the numbers from a previous incarnation of the database. ImportEngine will not allow any values for auto-id fields to pass through without being translated.

Another feature is that inserts will be re-ordered if necessary, such that a record which refers to an unknown key will be placed in a waiting list until that key is seen and inserted.

BUGS

Currently, there are situations where a record will get inserted before the required foreign constraint is met. For instance, if <C.id> has a FK of <B.id> which has a FK of <A.id>, then an insert for table C will be tried as soon as the corresponding key is added to table A. Some additional logic could prevent this situation, but sometimes circular dependencies show up which would prevent any records form getting inserted at all. Some fancy logic will be needed to solve this the "right way" (which may involve inserting partial records and updating them later, and in other cases might just be better handled by disabling constraints temporarily). In the meantime, we just try inserting things repeatedly until they all succeed or until no progress is made.

While DBIC supports fancy nested inserts and selects, we currently only translate keys for flat records. I would love to support the full DBIC semantics, but it will require much more time to implement.

  # Will not work yet!
  # record A will get mapped to the correct Foo, but A.category will not get translated.
  { action => 'insert', source => 'Category', data => { id => 'n', data => 'category of blah' } }
  { action => 'insert', source => 'Foo', data => { id => 1, a => { category => 'n', data => 'blah' } } }

Key translations do not happen for searches. Implementing this would require a full-blown search logic processor. Until this is implemented, do not write searches that depend on a generated key. However, the whole point of a search is to find generated IDs of existing records based on non-generated fields, so not being able to search on a generated field shouldn't be that big of a problem.

  # Will not work yet!
  # 'x' will not get translated to a Foo.id
  { action => 'find', source => 'Foo', search => { parent_id => 'x' }, data => { id => 'y' } }

And finally, I don't have support for multi-column keys. However, I left room in the design for this. We just don't happen to have any in our projects so far, so I didn't waste time implementing it.