STEADY STATE CHECKS

Sometimes ETLp processing of a file shouldn't begin until the file stops changing. This check is performed by repeatedly checking the size of the files and pausing for a specified time. When the file size doesn't change, then the item completes. This is typically used in conjunction with a File Watcher:

1. The watcher monitors a directory for the appearance of one or more files that match a pattern.

2. When one or more files appear, the watcher calls an iterative job to start processing the files.

3. Because the processing is triggered the sudden appearance of the file(s), it should not proceed while the files are deemed to be "live." (still being written to).

4. The first item in the iterative processing is a steady state check. It will wait indefinitely until the file is deemed "steady."

• name. The name of this item.

• interval. How long to wait (seconds) before checking whether the file size has changed. If the file is not steady, the plugin loops indefinitely, pausing for an interval length of time between each check. The default is five seconds.

• on_error. Override the setting for the job. See ETLp::Manual::Index for a detailed explanation of this parameter.

Example

    <item>
        name     = customer file check
        type     = steady_state_check
        interval = 30
    </item>