NAME - EEDB::Tools::MultiLoader

DESCRIPTION

MultiLoader is a single class which wraps up database specific optimizations for speeding up bulk loading. Since eeDB is primarily a data-mining system, most of the data is loaded in bulk. This loader augments the class specific ->store() functions. Currently eeDB supports sqlite and mysql databases and there are specific optimizations here for these two database engines.

AUTHOR

Jessica Severin <severin@gsc.riken.jp>

LICENSE

 * Software License Agreement (BSD License)
 * EdgeExpressDB [eeDB] system
 * copyright (c) 2007-2009 Jessica Severin RIKEN OSC
 * All rights reserved.
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *     * Redistributions of source code must retain the above copyright
 *       notice, this list of conditions and the following disclaimer.
 *     * Redistributions in binary form must reproduce the above copyright
 *       notice, this list of conditions and the following disclaimer in the
 *       documentation and/or other materials provided with the distribution.
 *     * Neither the name of Jessica Severin RIKEN OSC nor the
 *       names of its contributors may be used to endorse or promote products
 *       derived from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ''AS IS'' AND ANY
 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
 * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDERS BE LIABLE FOR ANY
 * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
 * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
 * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

APPENDIX

The rest of the documentation details each of the object methods. Internal methods are usually preceded with a _

flush_buffers

  Description  : MultiLoader uses internal FIFO buffers for speed. 
                 After all loading these buffers must be flushed to ensure they are empty
                 and all objects have been written
  Returntype   : none
  Exceptions   : none
  Example      :      my $multiLoad = new EEDB::Tools::MultiLoader;
                      $multiLoad->database($eeDB);
                      $multiLoad->do_store(1);
                      foreach my $mydata (@data_array) {
                        my $feature = new EEDB::Feature;
                        # do something with $mydata and $feature to create the feature
                        $multiLoad->store_feature($feature);
                      }
                      #after all store_xxxx methods, must flush any remaining objects from buffers
                      #usually the last thing we do in a script
                      $multiLoad->flush_buffers();
                      exit(1);
                      

store_feature

  Description  : MultiLoader uses internal FIFO buffers for speed. 
                 This method queues up a Feature into MultiLoader. When the buffer is full
                 it will do a bulk load, otherwise it will return directly.  If called
                 with no $feature parameter it will force a buffer flush and write.
  Parameter[1] : feature [EEDB::Feature] or undef to flush
  Returntype   : none
  Exceptions   : none
  Example      :      my $multiLoad = new EEDB::Tools::MultiLoader;
                      $multiLoad->database($eeDB);
                      $multiLoad->do_store(1);
                      foreach my $feature (@feature_array) {
                        $multiLoad->store_feature($feature);
                      }
                      $multiLoad->store_feature(); #no parameter so empties buffers
                      exit(1);
                      

update_feature_metadata

  Description  : this method will take a feature which was fetched from a database
                 and which may have new metadata/symbols and it will store that new
                 metadata/symbols and link them to the feature.
  Parameter[1] : feature [EEDB::Feature]
  Returntype   : none
  Exceptions   : none
  Example      :      my $multiLoad = new EEDB::Tools::MultiLoader;
                      $multiLoad->database($eeDB);
                      $multiLoad->do_store(1);
                      foreach my $feature (@feature_array) {
                         #
                         # do some metadata additions here
                         #
                         $multiLoad->update_feature_metadata($feature);
                      }
                      $multiLoad->update_feature_metadata(); #to flush the queues