Revision history for Perl extension Parallel::Depend
(most recent first).
4.10.1 Wed Aug 12 13:25:33 EDT 2009
Finished removing smart matches; which fixed
the broken tests.
4.09 Tue Jun 30 12:14:45 EDT 2009
- POD. It ain't perfect but does include the
"alias % global" and "ad_hoc" settings for
embedded schedules.
- share_queue without prepared queue is non-fatal,
returns empty list.
- Force and restart used to be exclusive, with
restart failing if any jobs looked like they
were currently running. In the interest of
offering more rope for your gun [bullets for
your gallows?], force+restart now treats
that any incomplete runfiles as non-running,
failed jobs and restarts them.
- $SIG{ INT } stores an abort message -- makes
it simpler to use ^C to abort a queue and then
restart it.
- In order to handle situations where the jobs
fork their own cleanup tasks on exit (e.g.,
gzip output), the reaping cycle loops via
next if the current job is not in the fork
table. This will reap however many orphans it
can in one pass without bothering to check
with runnable.
4.08 Sun Jun 28 18:52:54 EDT 2009
- Groups have to do some housekeeping on their
way out. Simplest way to ensure it gets done
is add an ad_hoc attribute to each group job.
Since the group's job runs at the end, running
it without forking allows any necessary
cleanups to happen (see next item also).
This has an advantage for long-lived
schedules since purging the extra data
keeps the queue at a reasonable size
if multiple sets of ad-hoc jobs are added
over time.
- ad_hoc groups are a convienent way for
factory items to insert schedule content.
Catch: The object that dispatched ad_hoc may
not be the one used to unalias the job within
execute.
Fix: the ad_hoc method adds the current object
into the attributes for its hidden group.
Within unalias, the ad_hoc_mgr attribute is
used if it exists to resolve method calls.
The object will stay alive for the group
execution due to being stored in the group's
attributes, which are purged on the way out
of the group (see previous item).
Result is that a main handler can use
something like:
my $child = $other_class->new;
$mgr->share_queue( $child );
...
$child->ad_hoc( @stuff );
and know that $child will be used to
dispatch the jobs added via ad_hoc
and also that $child will be destroyed
after all of its ad-hoc jobs have been
completed.
- Added "purge_group" to purge the attributes
and aliases for the group job itself and
the group-specific tables for jobs that
ran inside of the group.
This is called from the group method,
call this via SUPER or NEXT if the
method is overridden. This is where
the ad hoc manager object is dropped,
so failing to cause this will leave
any number of otherwise-short-lived
objects floating around in the attributes
until the schedule completes.
- Add "has_queue" with alias "was_prepared" to check if the
manager object has called prepare successfully -- which will
leave it with a queue.
4.07 Mon Jun 8 19:43:45 EDT 2009
- Fix typo "INT_" in signal name.
- Two things simplify generation of template
schedules: default alias and 'ad_hoc' attribute
for the alias.
- Add single-level attribute for groups to use the
group name as an alias for all jobs in the group.
This is for staged operations where a set of
modules handle, say, the processing of a group
of files in one dataset. They all have the same
methods for each stage:
serialize < Nodes = pass1 >
serialize < Names = pass1 >
serialize < Division = pass1 >
serialize < Delnodes = pass1 >
serialize < Merged = pass1 >
serialize < Nodes : >
serialize < Names : >
serialize < Division : >
serialize < Delnodes : >
serialize < Merged : >
becomes just:
serialize < alias % pass1 >
serialize < Nodes : >
serialize < Names : >
serialize < Division : >
serialize < Delnodes : >
serialize < Merged : >
The alias is not inherited to avoid accidentally
polluting ad_hoc or nested groups. This defines
a default alias, allowing for:
serialize < alias % pass1 >
serialize < Nodes = foo > # explicit alias is foo
serialize < Nodes : > # default alias is pass1
serialize < Names : >
serialize < Division : >
serialize < Delnodes : >
serialize < Merged : >
- The 'ad_hoc' attribute can be assigned to an alias.
This simplifies schedules like:
serialize < alias % pass1 >
serialize < Nodes ~ ad_hoc>
serialize < Names ~ ad_hoc>
serialize < Division ~ ad_hoc>
serialize < Delnodes ~ ad_hoc>
serialize < Merged ~ ad_hoc>
serialize < Nodes : > # default alias is pass1
serialize < Names : >
serialize < Division : >
serialize < Delnodes : >
serialize < Merged : >
to just:
serialize < alias % pass1 >
serialize < pass1 ~ ad_hoc>
serialize < Nodes : >
serialize < Names : >
serialize < Division : >
serialize < Delnodes : >
serialize < Merged : >
Which leaves each of the jobs (Nodes, Names, etc)
handed of as ad_hoc jobs via the managers 'pass1'
as $mgr->pass1( $job ).
- Pushed the alias resolver into P::D::Queue, unalias
returns the name and sub, but $que->resolve_alias( $job )
will return just the alias. This is used in merge_attrib
to get alias attributes.
- merge_attrib uses global, alias, and job attributes.
this allows for "alias ~ foo" in the schedules.
4.06 Thu Jun 4 16:14:16 EDT 2009
- Added ./Profile programs to MANIFEST. These are
stripped down schedules with empty jobs that
are useful for profiling the library. For example:
make && perl -Mblib -d:Profile Profile/many-jobs-fork;
less prof.out;
Note that these require Devel::Profile, which
is not in the dependency tree.
- Add boilerplate to ad_hoc jobs.
- Pushed the expensive tests into t/expensive.
They still require EXPENSIVE_TESTS true but
this avoids the extraneous junk.
- Add a few more ad_hoc tests [which seem to work].
- Pushed namespace, group name generation and splitting
of job_ids into queue in order to keep the constants
used with them ($nspace_sep, $job_id_sep) private.
4.05 Fri May 29 12:36:43 EDT 2009
- Split out que contents lookups into separate
module. Doesn't add any new functionality but
helps keep the code cleaner and should simplify
overloading storage of the contents via DBM::Deep
or DBIx modules.
- The queues "attrib" and "alias" values are
localized more simply using $que->alias and
$que->attrib.
- Saner names for a few of the internal methods:
job_attrib became merge_attrib.
- Removed unused 'group' from the queue structure.
- Test cleanups, in particular t/05* uses an autoload
to make the nesting aliases more obvious -- and
test the autoload feature.
- Job-specific attributes for anything pending are
cached to save unnecessary calls to $que->merge_attrib
(profiling showed it to be one of the big hits).
- Added a bit of boilerplate to the signal handlers
to avoid a logic race with a signal arriving
before the defaults had been restored in the child
processes.
- The runnable test became to expensive for
large jobs. The simplest speedup was to
replace empty hashes with false values
when the hashes are first found to be empty.
This changed the code from grep { ! %{ $h->{$k} } }
to grep { ! $h->{$k} } and seemed to help.
- There are also a few performance tweaks in
precheck, which only checks the logdir, rundir
once (via gendir) and uses them as-is for the
individual file checks.
- Documentation is [surprise] lagging. It isn't
hopeless but definitions of the guts are mostly
useless at this point -- though the interfaces
work as shown. Adding proper doc's for ad_hoc
schedules is next, then the guts.
4.04 Thu May 21 15:57:40 EDT 2009
- Added '~' to set job-specific attributes.
This avoids having to set jobs calling ad_hoc
in their own group in order to avoid forking
them. The nofork setting is requird to make
the schedule changes visible to the execution
handler (otherwise they are installed into a
forked proc's copy of the queue and discarded).
The new syntax also allows setting verbose,
etc, for single jobs. The default for most
items is true (1), separate the attribute
with whitespace to store another value:
peaches : cream
peaches ~ ad_hoc # peaches is *not* forked.
peaches ~ verbose # verbose % 1 for peaches.
cream ~ verbose 2 # verbose %2 for cream.
peaches = frobnicate # ad_hoc doesn't affect alias
- Begin [long overdue] POD updates.
- Isolate all attribute sanity checks in
validate_attrs. Makes it easier to
overload.
- Clean up t/13*t for checking ad_hoc handling.
4.03 Tue May 19 14:15:18 EDT 2009
- $mgr->ad_hoc will now add jobs into a running
schedule. This is intended for cases where one
job will, say, examine the filesystem and
generate jobs for each file it finds. Because
it is not possible [yet] to re-validated a
running queue, it is up to the caller to
avoid deadlocks in schedules with ad-hoc
components. See t/13*t for examples of
generating and using the ad_hoc call.
- Add "prefork" and "postfork" calls. The former
is made in the parent process, the latter in the
child. These are mainly for cleaning up things
like DBI handles to a known state. If the manager
object can( thing ) then it will be called.
4.02 May 11 2009
- Remove Devel::Size from the main tests; add new
ones that aren't part of the normal test cycle
for anyone else who cares. That removes Devel::Size
from the dependency list.
4.00 Mon Mar 23 11:01:50 EDT 2009
- use v5.10;
- Adding nested groups required a full re-write
of the guts. The groups are parsed completely,
including nesting, during prepare. This means
that there are no more delayed-effects from
parsing groups. This now allows the groups
to be handled without a separate fork, so the
maxjobs applies across multiple groups running
at the same time.
Keeping the group items in on place required
adding a namespace for jobs within groups --
the older method hid the groups from one
another via forks. This leaves the keys for
job, alias, attribute, etc, entries looking
like $que->{ $namespace, $job }. The default
(global) namespace is an empty string, successive
levels of groups are appended to their parent
namespace to get their own.
The namespace and name can be taken from:
my $i = rindex $fullname, $Parallel::Depend::job_id_sep;
my $namespace = substr $fullname, 0, $i;
my $name = substr $fullname, $i+2;
the separator variable is a read-only copy of
$; (which gets messy in syntax use).
Namespaces are also used to generate log and
runfiles, with the $; replaced with '-'.
- Same basic interface, saner logging,
nested groups. Also the parser and pieces of the
execution engine were broken out into smaller
chunks for easier viewing.
The main external differences are that the run
files include a timestamp and the log files
use a group-nested namespace for everything.
Logging also includes the namespace (i.e.,
nested group) for all jobs to uniquely
identify them (i.e. no more collisions between
job names in separate groups). Unneeded subs
were also stripped from P::D::Util (see
00* tests for quick list of public methods).
Dependencies between groups are also a bit
more reliable with group namespaces for the
jobs.
More, smaller tests give a better illustration
of how to use the code.
This also leaves the queue structure suitable
for persistence management via DBM::Deep, which
would allow simpler restarts without depending
on the local filesystem for bookkeeping.
3.07 Wed Jan 28 16:15:58 EST 2009
- Update META.yaml, minor cleanups.
- use fileparse to strip directory, suffix from
job before generating pidfile. this allows
rooted paths to be used as job specifiers.
- POD
3.06 Sun Jan 25 19:48:58 EST 2009
- Put autodie into Makefile.PL
3.05 Thu Jan 22 18:49:29 EST 2009
- Push the prefix into attributes to simplify handling
during group/subqueue. This puts all of the inherited
items into @inherit.
- Add tests for existing .pid, .out, .err files in
t/0[12]*t.
- Update POD for changes since Schedule::Depend was
retired in favor of "Parallel::Depend".
- Remove 'executed' flag: it caused too much confusion
when used with subque and groups. The precheck method
deals with this gracefully enough at this point.
3.04 Mon Jan 19 16:44:06 EST 2009
- Minor code & doc cleanups.
- Handle prefixes a bit more gracefully by
defaulting the initial prefix to basename $0.
3.03 Sat Dec 27 15:00:13 EST 2008
- Regex Typo.
3.02 Wed Dec 24 13:05:32 EST 2008
- Clean up group handling.
- Clean up whitespace handling.
- Simplify loops for building que elements prior
to breaking them out into separate sub's.
- Add autodie to requisites.
3.01 Tue Dec 23 12:59:47 EST 2008
- Modify regex to handle splitting up standalone
jobs ('foo:') to handle whitespace variances.
3.0 Mon Dec 22 18:05:42 EST 2008
- Reworked interface, parsing. Interface now uses
inside-out class for the que data, allows using
any generic object as the manager.
- Parsing input schedule is faster, especially for
large schedule lists.
2.5 Fri Apr 29 19:48:08 EDT 2005
- Added S::D::U::handle_que_args
Allows que methods to use:
my ( $que, $config ) = &handle_args;
or
my ( $config ) = &handle_args;
and yanks $que off of the stack. It also sets
$DB::single = 1 if $que->debug (i.e., automatic
breakpoint in debug mode) and logs the caller,
arguments.
This replaces the top few lines of nearly every
que method.
2.4 Tue Apr 26 16:54:56 EDT 2005
- Replaced use of internal "debug" que attribute with
"validate" to avoid conflict with "runsched" use of the
attribute (debugging a queue). This allows for:
$que->subque( %blah )->validate->execute;
when executing via runsched in debug mode.
2.3 Sat Jan 15 10:17:11 EST 2005
- Added "splat" to Utilities. This writes out larger items
as tab-separated-values files, which saves overhead of
Dumper. Update slurp to eval then split on newlines and
tabs (i.e., handle splat output).
2.2 Mon Jan 10 08:20:47 UTC 2005
- Added sanity check for missing classes after use base of
the user's use_base list. This spots missing classes (i.e,
ones that didn't compile) before their sub's fail up as
"shellexec" calls at runtime.
2.1 Sun Dec 26 06:34:50 EST 2004
- Fixed a typo in Utilities, replaced {basename} with {basenames} in
config check for localpath.
2.00 Sun Dec 26 06:34:50 EST 2004
- Added doc's for Execute, Utilities, and Config
modules. Updated comments in all of the modules.
1.00 Fri Apr 9 01:28:58 CDT 2004
- Added Parallel::Depend::Execute, S::D::Config,
- Added Parallel::Depend::Execute, S::D::Config,
S::D::Utilities to simplify generating #! code and
methods for schedules. Execute exports "runsched"
to handle running or debugging the schedules. The
user supplies one more module to define a defaults
hash and the #! code can be 4 lines.
- Utilities includes logging, nastygrams, progress
mail, directory sanity checks, local path generation.
- Config handles extracting relavant portions of the
main config hash. This is mainly for use in methods
since they can use "$que->moduleconfig" to get a
hash of defaults useful in the current module.
- The S::D interface is unlikely to change at this
point.
- 2do: The added modules need more POD and tests in
test.pl.
0.33 Mon Mar 15 23:07:33 CST 2004
- Added use of @::ttyz to fork handler for easier
debugging of multi-level forks. Pre-opened term's
can be listed in the global variable, which is
shifted off before each fork. This simplifies
multi-level debugs (e.g., for groups).
- execute does a better job of handling non-zero group
exits and die's. They are stored in the $que object
along with being handed back via exit status.
- Doc bugs.
0.31 Sat Jan 4 21:40:03 CST 2003
- Fixed noabort mode (works like "make -k"). Passing in
abort => 0 will not set the $que->{abort} value and
will thus skip only jobs which depend on the one that
failed.
- Comment cleanups.
0.30 Mon Nov 4 17:11:56 CST 2002
- Parallel::Depend::subque can also handle a hash referent
- Parallel::Depend::subque can also handle a hash referent
as the schedule object.
- ref $_[0] eq 'ARRAY' allows passing the queue in as an
array referent. Doesn't allow for blessed arrays, however,
which should stringify into the schedule.
- added "ttylog" to avoid re-opening STDERR/STDOUT to .out
and .err files. This puts everything from the scheule onto
the tty. Mainly useful for debugging or where network
interfaces might get cut off if there isn't any activitiy
on the connection for too long.
- ttylog mode does not overwrite any existing log files
(or create new ones). This avoids creating a directory
full of empty logs or overwriting valid ones during
a test run.
- Replaced the top-level que attributes (verbose, ttylog,
force, ...) with $que->{attrib}. This is the single unit
of inherited data for sub-queues.
- Added $que->{user} as inherited piece of sub-queues along
with $que->{attrib}. The "user" entry is not managed in any
other way by by S::D can be used to store information carried
into sub-qeues, blessed, whatever.
- Minor internal bugfixes.
- Updated test.pl to handle new que syntax.
0.29 Thu Sep 12 19:31:32 CDT 2002
- Parallel::Depend::prepare will now take a hash ref as
- Parallel::Depend::prepare will now take a hash ref as
the schedule argument. This allows the schedule to be
a list, text scalar, or hash ref.
0.28 Thu Sep 12 19:31:11 CDT 2002
- Nothing, dealt with an upload screwup.
0.27 Tue Sep 10 14:32:58 CDT 2002
- Doc, comment cleanups.
- Cleaned up the distribution package a bit (Depend.pm
now lives in lib/Schedule).
0.26 Thu Jun 6 09:06:45 CDT 2002
- debug argument avoids calling unalias, simply checks
if the job list deadlocks. This will >>not<< process groups
beyond determining if they deadlock w/in the main schedule.
The main purpose for this is assuring that the log and run
directories are in place and that the que doesn't obviously
deadlock.
- nofork argument runs the que serially without forking.
This is intended mainly for debugging and allows running
scheduled using "perl -d" with impunity. It effectively
limits the maxjobs to 1 since nothing is forked, the jobs
are executed in an eval one by one.
- In order to avoid conflicts with the accessor methods,
the "debug" sub has been renamed to "validate". If someone
thinks of a better name warn me.
- POD updates.
0.25 Wed May 22 02:17:08 CDT 2002
- Blocking on job slots displays the blocked jobs if verbose > 1.
- Replace $que->{abort} with $que->{failure} to avoid confusion
with $que->{noabort} switch value. Failed queues now have
the message in $que->{failure}. This doesn't effect the
external interface (aside from removing the "abort" accessor
method and replacing it with "failure").
- Added "force" argument to override precheck croaking
on non-empty pidfiles without an exit. Used with
restart, it forces execution of anything without an
exit status in the pidfile.
- Decided on group syntax. Using '::' for groups is
problematic due to Perl's module notation; almost
anything else might appear in a shell alias. Current
notation is to enclose the jobs in angle-brackets
("<>") surrounded by whitespace. The closing delimeter
makes it a bit more obvious that the group "encloses"
the jobs.
groupname < job1 job2 : job3 >
groupname < job4 : job5 job6 >
creates a single group with 6 jobs in it. Syntax for
the group is a full scheudule. The default handler is
group (i.e., groupname is aliased to "group" if none
already exists when the group is processed).
The group mechanism will look something like:
Create a lookaside list for the group with the group's
schedule in it. The schedule is keyed by the groupname.
When the group becomes runnable then $que->group('groupname')
is called, which prepares the schedule from the lookaside
list and then executes it. If the groupname is aliased to
another method then that one will be used to dispatch the
group's schedule (e.g., $que->mygroup('groupname') via
"groupname = mygroup" in the schedule).
The schedule then looks something like:
transform = group # optional, added automatically
transform < mungethis : mungethat >
transform < mungeother : >
transform : extract
load : transform
This creates a group with munge* jobs in it (they can be spread
out however is most useful). The group handler converts this into
something like:
transform = group
transform : extract
load : transform
When "transform" becomes runnable it is called via
$que->group( 'transform' ) which calles subque with
( sched => 'mungethis mungethat mungeother :' ) then
prepares executes the sub-que.
- More doc updates. They now reflect most of reality.
Included example of using multi-line entries in the
schedule via array ref.
- Fix typos in progress message trim, docs.
0.24
- unalias now returns an array of two items: the id string
and a closure. the string is used in execute for displaying
job status, the closure does the deed. This avoids the
current stringified code referent going into the pidfile
as the job identifier.
- Added "STUB" with behavior identical to PHONY (i.e., returns
$job (STUB) immediately). Only purpose is to improve documentation
during devleopment.
- Internal change to handle subque argument. This should not be
visible if the "subque" argument is not passed to preapre.
0.23 Wed May 15 11:12:44 CDT 2002
- Add'l sanity check in prepare for assigning to maxjob: arguments
only assign non-zero maxjob and only if maxjob is already zero.
This avoids the prepare aruments overriding a schedule limit. Since
zero is the default anyway this shouldn't cause any real pain.
- Minor boilerplate added to status sub.
- Comments.
0.22 Tue May 7 22:48:42 CDT 2002
- Added "informational" sub's to extract configuration and
status of object:
alias()
alias( @items )
returns a copy of the alias hash (not the
original) or a hash slice of the requested
items from the hash.
status()
Returns an anon. hash with the values of
alias(), restart, noabort, abort, verbose,
and debug keyed by their names.
These are mainly useful for creating sub-queues with
largely the same values as the parent.
And single-value items:
restart() true/false if que is restarting
noabort() true/false if que is running noabort.
abort() current abort value (string).
verbose() integer of verbosity.
debug() true/false if in debug mode.
jobz() $jobz{$pid} = $job (i.e., pid => job string ).
pidz() $pidz{$pid} = $pidfilepath
rundir() path of run directory
logdir() path of log directory
- Minor cleanups.
0.21 Sat May 4 13:43:47 CDT 2002
- PHONY jobs don't pre-check pidfiles.
- Zero-size pidfiles are assumed non-running in restart mode,
this saves having to blow off the unused pidfiles after a
que is aborted via ^C.
- Few minor '==' code bugs.
- ^C handling via $SIG{INT} handler setting $que->{abort}.
- TERM handler forwards a SIGTERM to the process group; forked
process sets $SIG{TERM} = 'DEFAULT'; if the child process
needs any speical SIGTERM handling it will install that for
itself on startup.
- Turned off warnings during the + 0 check in the child
process to avoid logging warnings about non-numeric
values.
- test.pl is in serious need of updates to check all of
this properly.
- Minor code, comment cleanups.
- Remove Depndency on Text::Balanced.
0.19 Sat Apr 20 22:27:38 CDT 2002
- At this point the interface isn't going to change.
- Clean up unalias a bit, pushed lexicals into the
if-ladder.
- Added croak ... unless $que at top of execute. Catches
accidental re-runs quicker.
- Code allows unalias to be called as a class method.
Main use of this is more automatic handling of the
if-ladder for cases where a preliminary job has to
be run outside of the schedule.
- Added eval to require the package for A::B-type
aliases. This guarantees that $packag->can( $subroutine )
succeeds even if S::D doesn't already have the
package available.
- Package::Subroutine alias handling includes an
eval to require the sub in case it hasn't already
been pulled in where unalias can see it.
- unalias always returns a code referent. handling of
system calls is pushed into S::D->shellexec. This
allows better logging of return values, since they
actually mean something speicific. Also allows
overloaded method to run better sanity checks
before dispatching to the shell if the environment
is better controlled.
- Slightly better testing in test.pl to account for
some of the more offball alias formats.
0.18 Fri Apr 20 2002
- Comments, POD.
0.17 Pretend this never happend...
0.16 Wed Apr 17 20:54:30 CDT 2002
- Updated &precheck to use the last line of a
pidfile if has >= 3 lines and perform a
numeric comparison to 0 for the exit status.
This allows for the child + parent both
writing the status and sub's returning strings
for successful execution.
- Put back the code that has the child writing
the returned scalar to the pidfile unmodified.
This will store any return strings from called
sub's. Parent gets back $result + 0 and writes
the numeric pic to the file. Either way, the
next pass with precheck can use "$linz[-1] = 0"
to determine if the job completed successfully.
- Removed Text::Balanced until I can find out
why it returns a block for "/foo/bar". Using
regex /^({.+})$/ to extract any blocks.
0.15 Tue Apr 16 12:13:59 CDT 2002
- Add the use of Text::Balanced::extract_codeblock
to handle anon sub's in the aliases. This allows
a schedule to contain valid perl blocks that will
be compiled and executed at runtime.
For example:
foo = { do_this($dir) || do_that($dir) or croak "$$: Neither that nor this!" }
...
foo : bar
Will call the do_this or do_that sub's with the
value of $dir expanded in the parent process at
runtime.
Note that the code must be on a single line.
Since unalias() is called in the parent this can
have side effects on subsequent children (e.g.,
foo = {do_this(pop @dirlist)}).
0.12 Fri Apr 12 14:05:41 CDT 2002
- Added handling for sub's named with packages for
the calls. e.g., "Foo::Bar::bletch : frobnicate"
will call the subroutine in that package if it
exists. This requirs some change in the parser
since it now has to detect colons that aren't
followed by colons to perform the split (not
difficult).
At this point unalias can automatically detect
methods, local subs and subs w/ package names
before dispatching to the shell.
Order of lookups is also changed, from Package::Sub
through method to sub.
- Replaced assignment for $run with test for ref $que
and defined $que->{alias}{$job}. This allows false
values for placeholders and avoids problems if unalias
is called as a class method.
0.11 Fri Apr 12 03:46:48 CDT 2002
- Stamped out some doc bugs. Mainly that the
required argument is "sched" not "depend",
empty aliases are no longer errors and
bogus pod.
0.10 Thu Apr 11 14:16:40 CDT 2002
- Update unlias to search for $que->can( $run ) or
__PACKAGE__->can($run)', using closures to call
the method/sub with the aliased string as an
argument. This allows scheduling of methods or
subs by name without any modification to the
S::D code.
The mechanism for finding sub's is to unalis the
jobname first, look for the ->can or CODE ref
and pass it the original jobname if it's found.
Thus:
abc : foo
xyz : foo
bar : foo
bar : abc xyz
to call foo('abc'), foo('xyz') and foo('bar').
This might be useful for, say, cleaning up multiple
directories. To pass more informaiton just use
the arguments as hash keys into a package/global
value with the extra info.
- Assigning an empty or 'PHONY' alias is a noop.
Thus:
foo =
or
foo = PHONY
Will insert an immediate return of zero job into
the schedule. This is mainly to neaten things up:
foo =
foo : long_named_job_one
foo : even_longer_named_job_two
foo : something_you_surely_do_not_want_to_type
bar : foo
now bar depends on all of the other three jobs without
the line-from-hell in the middle of a schedule. This can
also be handy for generating schedules on the fly, where
bar is a placeholder and the others are pushed onto a
stack if they are needed.
- Doc updates.
0.09
Put the tarball into a tarball rahter than
cpio archive, no change to the code.
0.08 Fri Apr 5 10:10:08 CST 2002
- Added runjob to handle the result of unalias
after the fork. If the unaliased scalar is a
code reference then it exits with the result
of &$job; otherwise exec's $job.
Using an unalias with:
no strict 'refs';
\&$job
to expand the tag from the scheule allows
queueing subroutines -- or a mixture of
subroutines and shell execs. This could also
return a closure to push evaluation parameters
even later into the cycle (or for testing).
See notes for unalias & runjob for examples.
- Added switch for handling failed jobs without
aborting the queue. This offers the same effect
as "make -k". If "noabort => true" is passed
into prepare then jobs that fail will have their
dependencies marked for skipping and the pidfiles
will get a nastygram + exit status of -1 (i.e.,
they will be re-executed on a restart).
The noabort code doesn't seem to break anything,
but has not been fully tested (yet).
- pidfile and output directories can be passed in with the arguments,
picked up from the environment or default to the current
executable's directory name. the environment method can be
handy for the single-argument version.
- Serious updates to pod and comments.
0.07
Default for $que->{alias}{logdir} and {rundir}
are "dirname $0". Simplifies running multpile
copies of the same file and schedule from
different directories via soft link.
Revised exit status writing. The child uses
system instead of exec and writes $? to its
own pidfile; the parent writes $? to the pidfile
if it is non-zero. This allows either the parent
or child to get zapped by a signal, leave the
other running and correctly record the status.
It also means that the pidfiles may be 4 lines
long on failure. Fix there is to read [0..2]
to check the status on the way in for restarts.
Updated comments to reflect reality a bit better.
0.06
Remove some of the extra newlines -- they aren't
required since individual job output goes to
stdout/stderr files.
Shuffled verobse prints to a bit to give saner
combinatins of output. At this point test.log
should give a reasonable idea of what slient,
progress and detailed output look like.
Process verbose as an alias for setting verbosity.
Add sanity check for odd number of arg's > 1
in prepare. Makes it harder to zap thyself by
adding "verbose => X" after a single-value
schedule entry.
test.pl checks for forkatotis in the module by
comparing the initial pid running test.pl with
what's running after the test_blah call; croaks
if the pid has changed since test.pl startup.
Updated comments, pod to reflect the changes
since 0.03.
Note: Still need to come up with a reasonable
definition for processing the debug alias/arg
during prepare and execution. It may require
debug levels like: 1 => don't fork, 2 => also
don't check or write pidfiles. Problem there
is making sure that mixing $que->debug with
$que->prepare( ... debug => X ) doesn't cause
unnecessary errors.
0.05:
Output of individual jobs goes to $logdir/$job.out
and $logdir/$job.err. Main purpose is to keep the
top-level schedule logs clean.
test.pl puts stdout to test.log -- saves a lotta
stuff flying by on make test.
verbose and debug arg's to prepare and execute
are independent (i.e, you can now debug in silent
mode and get minimal output).
0.04 Fri Mar 1 13:52:34 CST 2002
debug uses copy of queue, doesn't consume original
object during debug, returns original object if
debug is successful. see comments for use.
updated verbose handling, now has three levels: 0, 1, 2.
0 == stop/start messages and nastygrams, 1 == progress
messages from fork/reap; 2 == fairly detailed. $q->{verbose}
overrides the debug switch; no verbose setting w/ debug
gives verbose == 2. added description of changes to POD.
all verbose-controlled output goes to STDOUT, nastygrams
and que start/complete messages to STDERR.
doc updates to reflect changes in verbosity.
0.03 Wed Feb 27 12:20:18 CST 2002
Doc updates.
test.pl updated.
0.02 Wed Feb 6 17:25:02 CST 2002
Release-able version.
0.01 Wed Feb 6 10:20:32 2002
Beta