CSV_LOADER

The csv_loader plugin provides functionality to load delimited files into a database table. It relies on a control file to define the fields in the incoming data file. The following parameters must be set in the config section:

• table_name. The name table that the loader inserts the file data into.

• controlfile_dir. The directory that contains the control file.

• control_file. The name of the control file

The item parameters are:

• name. The name of this item.

• ignore_field_count. whether to continue if the number fields in the data (CSV) file don't match the number of fields in the control file.

• skip. The number of rows to skip in the file before loading begins. This allows header records to be ignored.

• localize. A boolean setting that instructs the loader to localize the end-of-line markers for the current file system. Defaults to 0.

• csv_options. A section containing additional options for processing. See http://search.cpan.org/dist/Text-CSV/lib/Text/CSV.pm#new_%28\%attr%29.

• on_error. Override the setting for the job.

Table Audit Column

The csv_loader plugin requires the table that stores the data have a column called file_id of type integer. This is used to link the file to its audit record.

Example

    <item>
        name = load customer file
        type = csv_loader
        # Skip one line
        skip = 1
        <csv_options>
            allow_whitespace = 1 
            sep_char         = | # use pipes not commas for delimiting data
        </csv_options>
    </item>