John Heidemann > Fsdb > Fsdb::Filter::dbsort

Download:
Fsdb-2.49.tar.gz

Dependencies

Annotate this POD

CPAN RT

Open  1
View/Report Bugs
Module Version: 2   Source  

NAME ^

dbsort - sort rows based on the the specified columns

SYNOPSIS ^

    dbsort [-M MemLimit] [-T TemporaryDirectory] [-nNrR] column [column...]

DESCRIPTION ^

Sort all input rows as specified by the numeric or lexical columns.

Dbsort consumes a fixed amount of memory regardless of input size. (It reverts to temporary files on disk if necessary, based on the -M and -T options.)

The sort should be stable, but this has not yet been verified.

For large inputs (those that spill to disk), dbsort will do some of the merging in parallel, if possible. The <--parallel> option can control the degree of parallelism, if desired.

OPTIONS ^

General option:

-M MaxMemBytes

Specify an approximate limit on memory usage (in bytes). Larger values allow faster sorting because more operations happen in-memory, provided you have enough memory.

-T TmpDir

where to put tmp files. Also uses environment variable TMPDIR, if -T is not specified. Default is /tmp.

<--parallelism> N

Allow up to N merges to happen in parallel. Default is the number of CPUs in the machine.

Sort specification options (can be interspersed with column names):

-r or --descending

sort in reverse order (high to low)

-R or --ascending

sort in normal order (low to high)

-n or --numeric

sort numerically

-N or --lexical

sort lexicographically

This module also supports the standard fsdb options:

-d

Enable debugging output.

-i or --input InputSource

Read from InputSource, typically a file name, or - for standard input, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

-o or --output OutputDestination

Write to OutputDestination, typically a file name, or - for standard output, or (if in Perl) a IO::Handle, Fsdb::IO or Fsdb::BoundedQueue objects.

--autorun or --noautorun

By default, programs process automatically, but Fsdb::Filter objects in Perl do not run until you invoke the run() method. The --(no)autorun option controls that behavior within Perl.

--help

Show help.

--man

Show full manual.

SAMPLE USAGE ^

Input:

    #fsdb cid cname
    10 pascal
    11 numanal
    12 os

Command:

    cat data.fsdb | dbsort cname

Output:

    #fsdb      cid     cname
    11 numanal
    12 os
    10 pascal
    #  | dbsort cname

SEE ALSO ^

dbmerge(1), dbmapreduce(1), Fsdb(3)

CLASS FUNCTIONS ^

new

    $filter = new Fsdb::Filter::dbsort(@arguments);

Create a new object, taking command-line arugments.

set_defaults

    $filter->set_defaults();

Internal: set up defaults.

parse_options

    $filter->parse_options(@ARGV);

Internal: parse command-line arguments.

setup

    $filter->setup();

Internal: setup, parse headers.

segment_start

    $self->segment_start(\@rows);

Sorting happens internally, to handle large things in pieces if necessary.

call $self-segment_start> to init things and to restart after an overflow $self-segment_overflow> to close one segment and start the next, and $self-segment_merge_finish> to put them back together again.

Note that we don't invoke the merge code unless the data exceeds some threshold size, so small sorts happen completely in memory.

Once we give up on memory, all the merging happens by making passes over the disk files.

segment_next_output

    $out = $self->segment_next_output($input_finished)

Internal: return a Fsdb::IO::Writer as $OUT that either points to our output or a temporary file, depending on how things are going.

segment_overflow

    $self->segment_overflow(\@rows, $input_finished)

Called to sort @ROWS, writing them to the appropriate place. $INPUT_FINISHED is set if all input has been read.

segment_merge_start

    $self->segment_merge_start($fn);

Start merging on file $FN. Fork off a merge thread, if necessary.

segment_merge_finish

    $self->segment_merge_finish();

Merge queued files, if any. Just call dbmerge(1) to do all the real work.

run

    $filter->run();

Internal: run over each rows.

AUTHOR and COPYRIGHT ^

Copyright (C) 1991-2008 by John Heidemann <johnh@isi.edu>

This program is distributed under terms of the GNU general public license, version 2. See the file COPYING with the distribution for details.

syntax highlighting: