DRAFT: Synopsis 17: Processes and Concurrency
Elizabeth Mattijsen <liz@dijkmat.nl> Audrey Tang <audreyt@audreyt.org> Christoph Buchetmann Tim Nelson <wayland@wayland.id.au>
Created: 13 Jun 2005 Last Modified: 10 Jul 2010 Version: 5
This draft document is a paste together from various sources. The bulk of it is simply the old S17-concurrency.pod, which dealt only with concurrency. Signals were added from S16-io, but haven't been merged with the conflicting S17 signals doco. An event loop section has been added here because a) Larry mentioned the idea, and b) Moritz suggested that http://www.seas.upenn.edu/~lipeng/homepage/unify.html be our model for concurrency, and in that model, an event loop underlies the threads.
An event loop underlies everything in this document. POSIX signals can interact with this, and concurrency is built on top of it. Naturally, IPC (inter-process communication) is documented here too (XXX or should be :) ).
The %*SIG variable contains a Hash of Proc::Signals::Signal.
class Proc::Signals::Signal { has $exception; # This specifies what exception will be raised when this signal is received has $interrupt; # See siginterrupt(3) has $blocked; # Is this signal blocked? cf. sigprocmask }
The @*SIGQUEUE array contains a queue of the signals that are blocked and queued.
The standard POSIX signals simply raise control exceptions that are handled as normal through the control signal handler, and caught by CONTROL blocks, as specified in S04.
To declare your main program catches INT signals, put a CONTROL block anywhere in the toplevel to handle exceptions like this:
CONTROL { when Error::Signal::INT { ... } }
The signals have defaults as specified in the table below. $blocked always defaults to false.
Signal Default Exception ------ ----------------- SIGHUP ControlExceptionSigHUP SIGINT ControlExceptionSigINT SIGQUIT ControlExceptionSigQUIT SIGILL ControlExceptionSigILL SIGABRT ControlExceptionSigABRT SIGFPE ControlExceptionSigFPE SIGKILL ControlExceptionSigKILL SIGSEGV ControlExceptionSigSEGV SIGPIPE ControlExceptionSigPIPE SIGALRM ControlExceptionSigALRM SIGTERM ControlExceptionSigTERM SIGUSR1 ControlExceptionSigUSR1 SIGUSR2 ControlExceptionSigUSR2 SIGCHLD ControlExceptionSigCHLD SIGCONT ControlExceptionSigCONT SIGSTOP ControlExceptionSigSTOP SIGTSTP ControlExceptionSigTSTP SIGTTIN ControlExceptionSigTTIN SIGTTOU ControlExceptionSigTTOU SIGBUS ControlExceptionSigBUS SIGPROF ControlExceptionSigPROF SIGSYS ControlExceptionSigSYS SIGTRAP ControlExceptionSigTRAP SIGURG Undefined SIGVTALRM ControlExceptionSigVTALRM SIGXCPU ControlExceptionSigXCPU SIGXFSZ ControlExceptionSigXFSZ SIGEMT ControlExceptionSigEMT SIGSTKFLT ControlExceptionSigSTKFLT SIGIO ControlExceptionSigIO SIGPWR ControlExceptionSigPWR SIGLOST ControlExceptionSigLOST SIGWINCH Undefined
A table below describes the exceptions.
Each of these has a default action as well. The possible actions are:
Term Default action is to terminate the process. Ign Default action is to ignore the signal ($signal.exception is undefined by default) Core Default action is to terminate the process and dump core (see core(5)). Stop Default action is to stop the process. Cont Default action is to continue the process if it is currently stopped.
Some actions do the Resumeable role. An exception listed in the table below that does the Resumeable role is marked with a * in the R column.
The exceptions are:
Signal Action R Comment ---------------------------------------------------------------------- ControlExceptionSigHUP Term ? Hangup detected on controlling terminal or death of controlling process ControlExceptionSigINT Term ? Interrupt from keyboard ControlExceptionSigQUIT Core ? Quit from keyboard ControlExceptionSigILL Core ? Illegal Instruction ControlExceptionSigABRT Core ? Abort signal from abort(3) ControlExceptionSigFPE Core ? Floating point exception ControlExceptionSigKILL Term ? Kill signal ControlExceptionSigSEGV Core Invalid memory reference ControlExceptionSigPIPE Term ? Broken pipe: write to pipe with no readers ControlExceptionSigALRM Term ? Timer signal from alarm(2) ControlExceptionSigTERM Term ? Termination signal ControlExceptionSigUSR1 Term ? User-defined signal 1 ControlExceptionSigUSR2 Term ? User-defined signal 2 ControlExceptionSigCHLD Ign * Child stopped or terminated ControlExceptionSigCONT Cont * Continue if stopped ControlExceptionSigSTOP Stop ? Stop process ControlExceptionSigTSTP Stop ? Stop typed at tty ControlExceptionSigTTIN Stop ? tty input for background process ControlExceptionSigTTOU Stop ? tty output for background process ControlExceptionSigBUS Core ? Bus error (bad memory access) ControlExceptionSigPROF Term ? Profiling timer expired ControlExceptionSigSYS Core ? Bad argument to routine (SVr4) ControlExceptionSigTRAP Core ? Trace/breakpoint trap ControlExceptionSigURG Ign ? Urgent condition on socket (4.2BSD) ControlExceptionSigVTALRM Term ? Virtual alarm clock (4.2BSD) ControlExceptionSigXCPU Core ? CPU time limit exceeded (4.2BSD) ControlExceptionSigXFSZ Core ? File size limit exceeded (4.2BSD) ControlExceptionSigEMT Term ? ControlExceptionSigSTKFLT Term ? Stack fault on coprocessor (unused) ControlExceptionSigIO Term ? I/O now possible (4.2BSD) ControlExceptionSigPWR Term ? Power failure (System V) ControlExceptionSigLOST Term ? File lock lost ControlExceptionSigWINCH Ign ? Window resize signal (4.3BSD, Sun)
See S04-control for details on how to handle exceptions.
XXX I'm unsure how the actions in the table above can be made to make sense. The Ign actions are already dealt with because %SIG{CHLD}.exception already defaults to undefined. The Term action will probably be self-solving (ie. will terminate the process). The others I'm just plain unsure about. XXX
XXX Everything about Alarm is from the old S17-concurrency.pod
An alarm is just a pre-arranged exception to be delivered to your program.
By the time alarm has arrived, the current block may have already finished executing, so you would need to set up CATCH blocks in places where an alarm can rise to handle it properly.
You can request an alarm using the number of seconds, or with a target date. It returns a proxy alarm object that you can do interesting things with.
multi Alarm *alarm (Num $seconds = $CALLER::_, &do = {die Sig::ALARM}, :$repeat = 1) multi Alarm *alarm (Date $date, &do = {die Sig::ALARM}, :$repeat = 1)
Perl 6's alarm has three additional features over traditional alarms:
alarm
One can set up multiple alarms using repeated alarm calls:
{ my $a1 = alarm(2); my $a2 = alarm(2); sleep 10; CATCH { also is critical; # if you don't want $a2 to be raised inside this when Sig::ALARM { ... } } }
To stop an alarm, call $alarm.stop. The alarms method for Conc objects (including process and threads) returns a list of alarms currently scheduled for that concurrent context.
$alarm.stop
alarms
When an alarm object is garbage collected, the alarm is stopped automatically. Within sink (void) context, the implicit alarm object can only be stopped by querying .alarms on the current process.
.alarms
We are not sure what alarm(0) would mean. Probably a deprecation warning?
alarm(0)
If you request a repeated alarm using the repeated named argument, it will attempt to fire off the alarm that many times. However, the alarm will be suppressed when inside a CATCH block that's already handling the exception raised by same alarm.
repeated
CATCH
To repeat 0 times is to not fire off any alarms at all. To repeat +Inf times is to repeat over and over again.
You can arrange a callback (like JavaScript's setTimeOut) in alarm, which will then be invoked with the then-current code as caller.
If you set up such a callback to another Conc object, what happens is just like when you called .die on behalf of that object -- namely, the callback closure, along with anything it referenced, is shared to the target Conc context.
.die
Unlike in Perl 5's ithreads where you cannot share anything after the fact, this allows passing shared objects in an ad-hoc fashion across concurrent parts of the program. Under the default (multiplexing) concurrency model, this is basically a no-op.
ad-hoc
This is a draft document. After being some time under the surface of Perl 6 development this is a attempt to document working concurrency issues, list the remaining todos and mark the probably obsolete and redundant points.
Concurrency can take many forms in Perl 6. With varying degrees of explicitness and control capabilities. This document attempts to describe what these capabilities are and in which form they can be accessed in Perl 6.
Concurrency comes in many shapes and forms. Most Perl users are used to the concept of a "process" or a "thread" (usually depending on the OS they work on). Some systems even are familiar with very lightweight threads called "fibers".
When discussing issues about concurrency with different people, it soon becomes apparent that everybody has his own set of "understandings" about what each word means, which doesn't make it any easier to describe Perl 6 concurrency.
It seemed the most natural to use the word "thread" to describe a process which has its own context, but also shares context with 0 or more concurrently running processes. Depending on your OS, or even specific version of your OS, this could still be a single "process" from the OS's point of view. Or it could contain an OS process for each thread. Or any mixture of these two implementations.
In this document we try to be agnostic about this: all we know in Perl 6 are "threads", which have their own context and share context with other concurrently running "threads". Whether they be process, threads or fibres at the OS level should not matter at the Perl 6 level.
And for sake of consistency, an unthreaded "normal" program is considered to be also running in a single thread.
In the past, there have been two models for concurrent processes in Perl. In general, these are referred to as "5.005 threads" (perldoc perlothrtut) and "ithreads" (perldoc perlthrtut).
perldoc perlothrtut
perldoc perlthrtut
The main difference between these two models from a programmer's point of view, is that variables in "5.005 threads" are shared by default. Whereas in the "ithreads" model, only variables that have been indicated to be "shared", are actually shared between threads. All other variable values are actually copies of the variable's value in the "parent" thread.
With regards to variables, the concurrency model of Perl 6 is closer to the "5.005 threads" model than it is to the "ithreads" model. In fact, all variables "visible" to a particular scope in Perl 6 will be accessible and modifiable (if allowed to do so) from all of the concurrent processes that start from that scope. In that sense, one could consider the "ithreads" model as a historical diversion: the Perl 6 concurrency picks up where the "5.005 threads" path left off.
(EM: maybe point out that the "ithreads" behaviour can be simulated with some kind of copy-on-write magic to be automagically added to all variable access inside a thread, except for those with an explicit "is shared" attribute?)
Differently from any current concurrent process implementation in Perl, there are no user accessible locks. Instead, the concept of Software Transactional Memory is used. This is in concept similar to the use of
BEGIN TRANSACTION ... do your uninterruptible actions COMMIT
in the database world. More interestingly, this also includes the concept of rollback:
BEGIN TRANSACTION ... do your stuff, but impossible to complete: ROLLBACK
This causes the state of the process to be reverted to the state at the moment the BEGIN TRANSACTION was executed.
Perl 6 supports this concept through contend blocks.
contend
These sections are guaranteed to either be completed totally (when the Code block is exited), or have their state reverted to the state at the start of the Code block (with the defer statement).
(EM: maybe point out if / how old style locks can be "simulated", for those needing a migration path?)
my ($x, $y); sub c { $x -= 3; $y += 3; $x < 10 or defer; } sub d { $x += 3; $y -= 3; $y < 10 or defer; } contend { # ... maybe { c() } maybe { d() }; # ... }
A Code block can be prefixed with contend. This means that code executed inside that scope is guaranteed not to be interrupted in any way.
The start of a block marked contend also becomes a checkpoint to which execution can return (in exactly the same state) if a problem occurs (a.k.a. a defer is done) inside the scope of the Code block.
The defer function basically restores the state of the thread at the last checkpoint and will wait there until an external event allows it to potentially run that atomic contend section of code again without having to defer again.
defer
If there are no external events possible that could restart execution, an exception will be raised.
The last checkpoint is either the outermost contend boundary, or the most immediate caller constructed with maybe.
maybe
The maybe statement causes a checkpoint to be made for defer for each block in the maybe chain, creating an alternate execution path to be followed when a defer is done. For example:
maybe { ... some_condition() or defer; ... } maybe { ... some_other_condition() or defer; ... } maybe { ... }
If placed outside a contend block, the maybe statement creates its own contend barrier.
Because Perl 6 must be able to revert its state to the state it had at the checkpoint, it is not allowed to perform any non-revertible actions. These would include reading / writing from file handles that do not support seek (such as sockets). Attempting to do so will cause a fatal error to occur.
seek
This will probably need to be expanded to all objects: any object that has some interface with data "outside" of the knowledge of the language (e.g. an interface with an external XML library) would also need to provide some method for freezing a state, and restoring to a previously frozen state.
If you're not interested in revertability, but are interested in uninterruptability, you could use the "is critical" trait.
sub tricky is critical { # code accessing external info, not to be interrupted } if ($update) { also is critical; # code accessing external info, not to be interrupted }
A Code block marked "is critical" can not be interrupted in any way. But since it is able to access non-revertible data structures (such as non-seekable file handles), it cannot do a defer as it would be impossible to restore the state to the beginning of the Code block.
Both "atomic" as well as "critical" propagate down the call chain. This means that any subroutine that in itself is not "atomic" or "critical" becomes uninterruptible if called inside a code block that is marked as "atomic" or "critical".
Atomic Code blocks called inside the call chain of a "critical" code block do not pose a problem, as they are more restrictive.
Any code that attempts to perform any non-revertible action (e.g. reading from a socket) will cause a fatal error when called inside the call chain of an Atomic Code block.
Coroutines are covered in S07
All outside of a thread defined variables are shared and transactional variables by default
Program will wait for _all_ threads. Unjoined threads will be joined at the beginning of the END block batch of the parent thread that spawned them
A thread will be created using the keyword async followed by a codeblock being executed in this thread.
async
my $thr = async { ...do something... END { } };
TODO: how you can access thread attributes inside a thread
async { say "my tid is ", +self; };
start time
end time
suspended (not diff from block on wakeup signal) waiting on a handle, a condition, a lock, et cetera otherwise returns false for running threads if it's finished then it's Nil
the CC currently running in that thread
TODO: IO objects and containers gets concurrency love!
$obj.wake_on_either_readable_or_writable_or_passed_time(3); # fixme fixme $obj.wake_on:{.readable} # busy wait, probably my @a is Array::Chan = 1..Inf; async { @a.push(1) }; async { @a.blocking_shift({ ... }) }; async { @a.unshift({ ... }) };
Stringify to something sensible (eg. "<Conc:tid=5>");
my $thr = async { ... }; say ~$thr;
Numify to TIDs (as in pugs)
my $thr = async { ... }; say +$thr;
TODO: Enumerable with Conc.list
TODO: Conc.yield (if this is to live but deprecated, maybe call it sleep(0)?)
sleep() always respects other threads, thank you very much
wait for invocant to finish (always item cxt)
my $thr = async { ... }; $thr.join();
throw exception in the invocant thread
set up alarms
query existing alarms
pause a thread; fail if already paused
revive a thread; fail if already running
survives parent thread demise (promoted to process) process-local changes no longer affects parent tentatively, the control methods still applies to it including wait (which will always return Nil) also needs to discard any atomicity context
TODO:
method throttled::trait_auxiliary:<is> ($limit=1, :$key=gensym()) { # "is throttled" limits max connection to this Code object # the throttling is shared among closures with the same key # the limit may differ on closures with the same key. # if the counter with the "key" equals or exceeds a closure's limit, # the closure can't be entered until it's released # (this can be trivially implemented using contend+defer) } class Foo { method a is throttled(:limit(3) :key<blah>) { ... } method b is throttled(:limit(2) :key<blah>) { ... } } my Foo $f .= new; async { $f.a } async { $f.b }
TODO document
Live in userland for the time being.
### INTERFACE BARRIER ### module Blah; { also is atomic; # contend/maybe/whatever other rollback stuff # limitation: no external IO (without lethal warnings anyway) # can't do anything irreversible also is critical; # free to do anything irreversible # means "don't interrupt me" # in system with critical section, no interrupts from # other threads will happen during execution # you can't suspend me my $boo is export; $boo = 1; # We decree that this part forms the static interface # it's run once during initial compilation under the # Separate Compilation doctrine and the syms sealed off # to form part of bytecode syms headers %CALLER::<&blah> = { 1 }; # work - adds to export set die "Eureka!" if %CALLER::<$sym>; # never dies # BEGIN { $boo = time }; sub IMPORT { # VERY DYNAMIC! our $i = time; %CALLER::<&blah> = { 1 }; # work - adds to export set die "Eureka!" if %CALLER::<$sym>; # probes interactively } } ### INTERFACE BARRIER ###
"I/O Considerations" in S16
"File Descriptors" in S16
"Sockets" in S16
Please post errors and feedback to perl6-language. If you are making a general laundry list, please separate messages by topic.
To install Perl6::Doc, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Perl6::Doc
CPAN shell
perl -MCPAN -e shell install Perl6::Doc
For more information on module installation, please visit the detailed CPAN module installation guide.