$Id: Options.pod,v 1.6 2004/03/25 19:27:59 tower Exp $
=begin html
<style = "text/css">
code { font-family: sans-serif; font-weight:bold; }
</style>
=end html
=head1 Setting Options
C<Net::AsycnZ> sets options by means of named parameters for both the parent process
and each of its child processes. Options for the parent are set in C<Net::AsycnZ-E<gt>new>.
Options for the child processes are set via the C<options> parameter of C<Net::AsycnZ-E<gt>new>;
the value of this parameter must be an array of C<Net::Z3950::AsyncZ::Options::_params>
objects.
If a C<_params> object doesn't exist for a child process, C<Net::AsycnZ-E<gt>new>
will create it with a set of default options. There will always be a C<_params>
object for every server in the C<servers> array, and they are cross-indexed, that is
C<$_params_object[0]> is used for C<$server[0]>, etc. So, if you are creating your
own array of C<_params> objects, you must keep this parallelism in mind.
=head2 Types of Options
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[1]
Options set in C<Net::Z3950::AsyncZ::new> which control the parent process
and selected features of the child processes for which no alternatives
are present: the alternatives are set as indicated in [2] and [3].
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[2]
Options set in a C<Net::Z3950::AsyncZ:Options::_params> object: this is returned
by C<Net::Z3950::AsyncZ::asyncZOptions()>. There is one C<_params> object
for each server: if you don't create one, it is created for you with
the default values. If you don't create a C<_params> object for
a server, then C<log> and C<query> options set in the C<AsyncZ>
constructor will be used. The rationale behind this is that you usually will
be asking one question across all servers and will usually be using only
one log file for debugging.
But in all other cases where it is possible
to set an option for the child in both the C<AsyncZ> constructor or
C<_params>, the C<_params> setting will be used. At the moment this affects
the C<format> and C<num_to_fetch> options.
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[3]
Options set in the C<Net::Z3950::Manager> by using the C<Z3950_options>
option of the C<_params> object. These take precedence over any others
and must be passed in with the first C<_params> object, that is, C<$_params_object[0]>,
because AsyncZ uses only one C<Net::Z3950::Manager>. The Manager
is created when setting up the first server passed into the constructor.
=over 4
=item Note:
Default values for options are shown to the right of the C<=E<gt>>operator:
HTML=>0
In some instances, the I<type> of variable is shown and defaults detailed in commentary:
format=>\&format
=back
=head2 Net::Z3950::AsyncZ::new
=over 4
=item cb
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<cb=E<gt>\&cb>
E<nbsp>E<nbsp>E<nbsp>callback function to which records will be sent as available.
See L<Output Callback|/"Output Callback (required)">.
=item format
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<format=E<gt>\&format>
E<nbsp>E<nbsp>E<nbsp>callback function to format individual lines of records.
See L<Format Callback|/"Format Callback (not required)">.
If you create a C<_params>
object for a server and do not set its C<format> option, then the default
C<format> will be used, even if you set the C<format> option of the
C<AsyncZ> constructor to another value.
=item interval
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<interval=E<gt>1>
E<nbsp>E<nbsp>E<nbsp>Event loop timer interval in seconds: This controls how frequently AsyncZ checks to
see if servers have responded and if the C<timeout> period is up.
=item log
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<log=E<gt>undef>
E<nbsp>E<nbsp>E<nbsp>controls how extended error messages are handled. There are two sets of
error messages--those handled through Net::Z3950::AsyncZ::ErrMsg and which are
meant for the user and those meant for debugging. The latter are generated by both
AsyncZ and the Perl library and can accumlulate at a rapid clip. AsyncZ writes
its debugging messages to STDOUT, while those coming from library
routines almost always go to STDERR. There are 3 options for C<log>.
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[1] C<undef>, the default, in which case all debugging messages
go to the terminal, and those written to STDOUT will end up in a browser if you are
on the web.
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[2]C<log=E<gt>Net::Z3950::AsyncZ::Errors::suppressErrors()> (or C<log=E<gt>suppressErrors()>
if you import the function)--in which case these messages will be suppressed
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>[3]C<log=E<gt>$filespec>,
in which case all of these messages will go to the file specified
in C<$filespec>
The C<Net::Z3950::AsyncZ::Options::_param> object also has a C<log> option--which means that you can
specify a log file for each child process--ie. for each server queried--
while keeping a separate one for the parent. Or you can set up a system where parent
and child_1 write to log.1, while child_2 and child_3 write to log.2, etc.
I<Note>: All error logs are automatically opened and closed.
I<Do B<NOT> open or close them yourself!>
Do B<NOT> open or close log files yourself!
=item maxpipes
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<maxpipes=E<gt>4>
E<nbsp>E<nbsp>E<nbsp>maximum number of forks to be executed at one time--the greater
the number the more resources are used--both of memory and cpu.
=item monitor
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<monitor=E<gt>0>
E<nbsp>E<nbsp>E<nbsp>timeout in seconds for a monitoring child process, or 0,
in which case a monitor is not set.
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>The monitor is a child process which
runs a timer and kills the parent process, if it exceeds the timeout period.
You run the monitor only if your software hangs.
An orderly shutdown of all runnning processes is put into effect,
the purpose of which is to prevent the development of zombie processes and to
release all shared memory.
=item num_to_fetch
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<num_to_fetch-E<gt>5>
E<nbsp>E<nbsp>E<nbsp>number of records to fetch; this setting will be used
only if you have not created a C<_params> object. This means that if you
create C<_params> object for the server and do not set its C<num_to_fetch>
option, then C<num_to_fetch> will default to 5 even if you have set another value
for C<num_to_fetch> in the C<AsyncZ> constructor.
=item options
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<options=E<gt>\@options>
E<nbsp>E<nbsp>E<nbsp>reference to an array of references to L</"Net::Z3950::AsyncZ::Options::_params"> objects.
Each reference is obtained from a call to L</"Net::Z3950::AsyncZ::asyncZOptions">. For instance:
@options = (
asyncZOptions(option_1=>opt_1,option_2=>opt_2, . . .),
undef,
asyncZOptions(option_1=>opt_1,option_2=>opt_2, . . .)
);
This array parallels the C<servers> array:
@servers = (
[$host_1, $port_1, $database_1],
[$host_2, $port_2, $database_2],
[$host_3, $port_3, $database_3]
);
C<$options[0]> is used for C<$server[0]> and C<$options[2]> for C<$server[2]>. If a
C<_params> object is not found or if it is not defined, as for $server[1],
then a default C<_params> object is created for the server.
=item query
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<query=E<gt> undef>
E<nbsp>E<nbsp>E<nbsp>the query string: its format depends on Z3950 querytype
and defaults to 'prefix' (as in C<Net::Z3950>). You can set
a separate Z3950 querytype for each query, or you can change the querytype
for all servers by using C<Z3950_options>.
If you create a C<_params> for a server but do not set the C<query> option
in C<_params>, then this C<query> will be used. This means that you can
set one C<query> for all of your servers without having to re-set it for
each of the C<_params> objects you create. But if you create
a C<_params> with a different C<query>, then the query set in C<_params>
will be used.
=item servers
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<servers=E<gt>\@servers>
E<nbsp>E<nbsp>E<nbsp>array of references to servers in form: [ $host, $port, $database]
E<nbsp>E<nbsp>E<nbsp>See C<options> above and C<AsyncZ.pod: "The Basic Script">.
=for html See also <a href="AsyncZ.html#the_basic_script">basic.pl</a>
=item swap_attempts
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<swap_attempts=E<gt>5>
E<nbsp>E<nbsp>E<nbsp>the number of times that a swap check will be done before exiting;
see C<swap_check> for details.
=item swap_check
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<swap_check=E<gt>0>
E<nbsp>E<nbsp>E<nbsp>the number of seconds between checks for swapping activity--
used when querying a great number of servers and requesting large amounts of data. It
instructs AsyncZ to sleep for C<swap_check> number of seconds before processing any
further connections. If you are attempting to process too much data for the size of your RAM,
the system will have to swap out of memory into the swap space on your disk; too much swapping causes
loss of data and disk "thrashing"--i.e. repeated disk access--and will overburden the system.
When C<swap_check> is set, AsyncZ will check for signs of swap activity; if it finds swap
activity it will go to sleep for the number of seconds set in C<swap_check> and then re-check
for C<swap_attempts> number of times. If the swap activity continues beyond this number
of checks, AsyncZ dies. For large throughput, you will probably want to set the monitor,
and to set it for a long period of time, for instance, 3000 seconds. This means that
you can set C<swap_check> to a period of 10,20, 30 seconds. The values you set on these
variables will depend on your own system memory resources and the amount of data you
are processing.
I<Note:> This has been tested only on Linux but should also work
on Unix, at least on Solaris.
=item timeout
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<timeout=E<gt>25>
E<nbsp>E<nbsp>E<nbsp>total timeout in seconds for all processes to complete their work.
=item timeout_min
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<timeout_min=E<gt>5>
E<nbsp>E<nbsp>E<nbsp>minumum timeout in secs to exit Event loop if all processes are finished;
a security blanket to make sure all processes get a chance to report their
results to the parent process before exiting the loop.
=back
=over
=item E<nbsp>
E<nbsp>E<nbsp>E<nbsp>
=back
=head2 Net::Z3950::AsyncZ::Options::_params
Where a C<_param> option duplicates an C<AsyncZ::new> option, consult the
C<AsyncZ::new> description for more details.
=over 4
=item HTML
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<HTML=E<gt>0>
E<nbsp>E<nbsp>E<nbsp>if true use default HTML formatting for records, if false format as plain text;
see L</"Row Formatting Priorities">.
=item Z3950_options
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<Z3950_options=E<gt>undef>
E<nbsp>E<nbsp>E<nbsp>reference to hash of additional Z3950 options.
These options are passed to the Z3950 Manager and
take precedence over C<_param> options and options set in
C<Net::Z3950::AsyncZ-E<gt>new>.E<nbsp>E<nbsp>
C<Z3950_options> makes it possible to implement Z3950 options which may not be specifically
accounted for in any of the options to the AsyncZ module. For instance, to ask for
"full" as opposed to "brief" records (which is the Z3950 default):
@options = (asyncZOptions(Z3950_options=>{elementSetName =>'f'}) <, (asyncZOptions(. . .), . . >);
I<Note:> To use this option, it must appear in the first C<_params> object of the C<_params> array,
C<$options[0]>, as in the above example. It is ignored in any subsequent uses. This
means that you cannot set these options on a per-server basis; they apply
across to board to all the servers you are querying. In the above exmaple, for instance,
you could not ask for brief records from some servers and full from others.
See L</"Types of Options">
=item cb
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<cb=E<gt>\&cb>
E<nbsp>E<nbsp>E<nbsp>reference to callback function to which records will be sent as available
=item format
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<format=E<gt>\&format>
E<nbsp>E<nbsp>E<nbsp>reference to a callback function that formats each row of a record
=item interval
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<interval=E<gt>5>
E<nbsp>E<nbsp>E<nbsp>timer interval for this forked process. See C<interval>
above under C<Net::Z3950::AsyncZ::new>.
=item log
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<log=E<gt>undef>
E<nbsp>E<nbsp>E<nbsp>controls how extended error messages are handled for this
child process. A separate log file can be opened for each process.
I<Note>: All error logs are automatically opened and closed.
See C<log> above under C<Net::Z3950::AsyncZ::new>.
=item marc_fields
=item marc_subst
=item marc_userdef
=item marc_xcl
These options are fully described and illustrated in Report.pod
under the heading "MARC Bibliographic Format".
=begin html
<br>
For the HTML version goto <A href="Report.html#marc_bibliographic_format">MARC Bibliographic Format</a>
in Report.html.
=end html
=item num_to_fetch
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<num_to_fetch=E<gt>5>
E<nbsp>E<nbsp>E<nbsp>number of records to fetch from this server.
=item pipetimeout
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<pipetimeout=E<gt>20>
E<nbsp>E<nbsp>E<nbsp>timeout in seconds for this child process
=item preferredRecordSyntax
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<preferredRecordSyntax=E<gt>Net::Z3950::RecordSyntax::USMARC>
E<nbsp>E<nbsp>E<nbsp>the Z3950 preferredRecordSyntax for this child process
=item query
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<query=E<gt>undef>
E<nbsp>E<nbsp>E<nbsp>the query for this process
=item querytype
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<querytype=E<gt>'prefix'>
E<nbsp>E<nbsp>E<nbsp>Z3950 querytype for this child process; it can be set to'ccl', or 'ccl2rpn'.
=item raw
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<raw=E<gt>0>
E<nbsp>E<nbsp>E<nbsp>(boolean) if true the raw record data for this process is returned; its format
is dependent on the C<render> option.
=item render
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<render=E<gt>1>
E<nbsp>E<nbsp>E<nbsp>(boolean) if C<true> the raw record data for this process is returned filtered
through the Z3950 C<Record::render> function; this is the default. If C<false> the
raw data is returned unfiltered in its original state. The unfiltered raw data can
be read using C<Net::Z3950::AsyncZ::prep_Raw> and C<Net::AsyncZ::get_ZRawRec>.
=item startrec
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<startrec=E<gt>1>
E<nbsp>E<nbsp>E<nbsp>number of the record with which to start result from Record Set.
=item utf8
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>C<utf8=E<gt>0>
E<nbsp>E<nbsp>E<nbsp>when set to C<true> conversions will be made to C<utf8/unicode>
characters from the character codes used in MARC records to represent non-latin1
and accented latin1 chatacters. When ouputting C<utf8>, you must call C<binmode>
on the ouput stream, for example:
binmode(STDOUT, ":utf8");
When outputting to a browser, you should also notify the browser:
print "Content-type: text/html;charset=utf-8'\n\n";
print '<head><META http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body>';
See the sample script: C<MARC_HTML.pl>.
I<Note:> To use C<utf8> you must have the C<MARC::Charset> module installed. Otherwise,
the C<utf8> option will be ignored.
=back
=over 4
=item E<nbsp>
E<nbsp>
=back
=head2 Row Formatting Priorities
If more than one option is set that affects the formatting of a record's rows, the following
priority squence is in effect:
raw, format, HTML, plaintext (default)
=over 4
=item E<nbsp>
E<nbsp>
=back
=head2 Methods for Setting _params Options
=head3 get/set methods
C<Net::Z3950::AsyncZ::Options::_params> provides a full range of C<get_option> / C<set_option> methods,
enabling the dynamic setting of option values.
$_params_object->set_HTML(0);
$num_to_fetch = $_params_object->get_num_to_fetch();
In addition there are functions for setting options with fixed values:
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>Function
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>
E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>E<nbsp>Equivalent
set_marc_xtra() set_marc_fields($Net::Z3950::AsyncZ::Report::xtra)
set_marc_all() set_marc_fields($Net::Z3950::AsyncZ::Report::all)
set_marc_std() set_marc_fields($Net::Z3950::AsyncZ::Report::std)
set_raw_on() set_raw(1)
set_raw_off() set_raw(0)
set_plaintext() set_HTML(0)
set_HTML() set_HTML(1)
set_prefix() set_querytype('prefix')
set_ccl=>() set_querytype('ccl')
set_GRS1() set_preferredRecordSyntax(Net::Z3950::RecordSyntax::GRS1)
set_USMARC() set_preferredRecordSyntax(Net::Z3950::RecordSyntax::USMARC)
The get/set methods guarantee that you have in fact set or queried the option you are interested in
and, in the case of the fixed value options, that you have set it to the value required. You don't
have to be concerned that a meaningless hash key will spring into existence through misspelling:
$_params_object = asyncZoptions(leg=>Error.LOG, num_to_fish=>3);
In the case of the some of the fixed value methods, one advantage is the obvious simplicity of calling C<set_GRS1()> instead of
C<set_preferredRecordSyntax(Net::Z3950::RecordSyntax::USMARC)>.
=head3 Net::Z3950::AsyncZ::Option::_params::option
This method works to both get and set values.
$value = $_params_obj->option('option');
$old_options_ref = $_params_obj->option(option=>value,option=>value,option=>value. . . );
B<params>
in get mode: 'option' to be queried
in set mode: list of option=>value pairs to be set (or %hash)
B<returns>
in get mode: $value of option being queried
in set mode: $old_options_ref -- reference to a hash of option=>value pairs
which have been replaced by list or %hash
=head3 Net::Z3950::AsyncZ::Option::_params::validOption
$bool = $_params_obj->validOption('option');
=head3 Net::Z3950::AsyncZ::Option::_params::invalidOption
$bool = $_params_obj->invalidOption('option');
Both of the above methods will enable you to determine whether an option you choose to
set is a valid option. Useful when using C<Net::Z3950::AsyncZ::Option::_params::option>.
$option = 'num_to_fetch';
$_params_obj->validOption($option) ? $_params_obj->option($option=>3) :
die "invalid option: $option";
=head3 Net::Z3950::AsyncZ::Option::_params::test
$_params_obj->test();
Calling this function will print a listing of I<defined> options and values for
C<$_params_obj>.
=over 4
=item E<nbsp>
E<nbsp>
=back
=head1 AUTHOR
Myron Turner
E<lt>turnermm@shaw.caE<gt>
or E<lt>mturner@ms.umanitoba.caE<gt>
=head1 COPYRIGHT AND LICENSE
Copyright 2003 by Myron Turner
This library is free software; you can redistribute it and/or modify
it under the same terms as Perl itself.
=for html
<br><br><br>
=begin comment
my %_fn_equiv = (
# functions requiring user parameters or returning values
pipetimeout=>undef,
interval => undef,
format =>undef,
HTML => undef,
raw => undef,
startrec =>undef,
num_to_fetch =>undef,
marc_fields =>undef,
marc_xcl =>undef,
marc_subst=>undef,
marc_userdef =>undef,
query=>undef,
log=>undef,
cb=>undef,
querytype =>undef,
preferredRecordSyntax=>undef,
Z3950_options=>undef
);
my %_fn_predef = (
# functions using predefined parameters
marc_xtra => [ 'marc_fields', $Net::Z3950::AsyncZ::Report::xtra ],
marc_all => [ 'marc_fields', $Net::Z3950::AsyncZ::Report::all ],
marc_std => [ 'marc_fields', $Net::Z3950::AsyncZ::Report::std ],
raw_on => [ 'raw', 1 ],
raw_off => [ 'raw', 0 ],
plaintext=> [ 'HTML', 0 ],
HTML=> [ 'HTML', 1 ],
prefix => [ 'querytype', 'prefix'],
ccl=> [ 'querytype', 'ccl'],
GRS1=> [ 'preferredRecordSyntax', Net::Z3950::RecordSyntax::GRS1],
USMARC=> [ 'preferredRecordSyntax', Net::Z3950::RecordSyntax::USMARC]
);
=end comment