PDL::Primitive - primitive operations for pdl
This module provides some primitive and useful functions defined using PDL::PP and able to use the new indexing tricks.
See PDL::Indexing for how to use indices creatively. For explanation of the signature format, see PDL::PP.
use PDL::Primitive;
Copyright (C) Tuomas J. Lukka 1997 (lukka@husc.harvard.edu). Contributions by Christian Soeller (c.soeller@auckland.ac.nz), Karl Glazebrook (kgb@aaoepp.aao.gov.au), Craig DeForest (deforest@boulder.swri.edu) and Jarle Brinchmann (jarle@astro.up.pt) All rights reserved. There is no warranty. You are allowed to redistribute this software / documentation under certain conditions. For details, see the file COPYING in the PDL distribution. If this file is separated from the PDL distribution, the copyright notice should be included in the file.
Inner product over one dimension
c = sum_i a_i * b_i
', BadDoc => 'If a() * b()
contains only bad data, c()
is set bad. Otherwise c()
will have its bad flag cleared, as it will not contain any bad values.', ); # pp_def( inner )
pp_def( 'outer', HandleBad => 1, Pars => 'a(n); b(m); [o]c(n,m);', Code => 'loop(n,m) %{ $c() = $a() * $b(); %}', BadCode => 'loop(n,m) %{ if ( $ISBAD(a()) || $ISBAD(b()) ) { $SETBAD(c()); } else { $c() = $a() * $b(); } %}', Doc => ' =for ref
outer product over one dimension
Naturally, it is possible to achieve the effects of outer product simply by threading over the "*
" operator but this function is provided for convenience.
'); # pp_def( outer )
pp_addpm(<<'EOD'); =head2 x
Signature: (a(i,x), b(z,i),[o]c(x,z))
Matrix multiplication
PDL overloads the x
operator (normally the repeat operator) for matrix multiplication. The number of columns (size of the 0 dimension) in the left-hand argument must normally equal the number of rows (size of the 1 dimension) in the right-hand argument.
Row vectors are represented as (N x 1) two-dimensional PDLs, or you may be sloppy and use a one-dimensional PDL. Column vectors are represented as (1 x N) two-dimensional PDLs.
Threading occurs in the usual way, but as both the 0 and 1 dimension (if present) are included in the operation, you must be sure that you don't try to thread over either of those dims.
EXAMPLES
Here are some simple ways to define vectors and matrices:
perldl> $r = pdl(1,2); # A row vector perldl> $c = pdl([[3],[4]]); # A column vector perldl> $c = pdl(3,4)->(*1); # A column vector, using NiceSlice perldl> $m = pdl([[1,2],[3,4]]); # A 2x2 matrix
Now that we have a few objects prepared, here is how to matrix-multiply them:
perldl> print $r x $m # row x matrix = row [ [ 7 10] ] perldl> print $m x $r # matrix x row = ERROR PDL: Dim mismatch in matmult of [2x2] x [2x1]: 2 != 1 perldl> print $m x $c # matrix x column = column [ [ 5] [11] ] perldl> print $m x 2 # Trivial case: scalar mult. [ [2 4] [6 8] ] perldl> print $r x $c # row x column = scalar [ [11] ] perldl> print $c x $r # column x row = matrix [ [3 6] [4 8] ]
INTERNALS
The mechanics of the multiplication are carried out by the matmult method.
Signature: (a(i,x),b(z,i),[o]c(x,z))
Matrix multiplication
We peruse the inner product to define matrix multiplication via a threaded inner product.
For usage, see x, a description of the overloaded 'x' operator
Cross product of two 3D vectors
After
$c = crossp $a, $b
the inner product $c*$a
and $c*$b
will be zero, i.e. $c
is orthogonal to $a
and $b
Threaded Index Add: Add a
to the ind
element of sum
, i.e:
sum(ind) += a
Simple Example:
$a = 2; $ind = 3; $sum = zeroes(10); indadd($a,$ind, $sum); print $sum #Result: ( 2 added to element 3 of $sum) # [0 0 0 2 0 0 0 0 0 0]
Threaded Example:
$a = pdl( 1,2,3); $ind = pdl( 1,4,6); $sum = zeroes(10); indadd($a,$ind, $sum); print $sum."\n"; #Result: ( 1, 2, and 3 added to elements 1,4,6 $sum) # [0 1 0 0 2 0 3 0 0 0]
1d convolution along first dimension
$con = conv1d sequence(10), pdl(-1,0,1), {Boundary => 'reflect'};
By default, periodic boundary conditions are assumed (i.e. wrap around). Alternatively, you can request reflective boundary conditions using the Boundary
option:
{Boundary => 'reflect'} # case in 'reflect' doesn't matter
The convolution is performed along the first dimension. To apply it across another dimension use the slicing routines, e.g.
$b = $a->mv(2,0)->conv1d($kernel)->mv(0,2); # along third dim
This function is useful for threaded filtering of 1D signals.
Compare also conv2d, convolve, fftconvolve, fftwconv, rfftwconv
EOD Pars => 'a(m); kern(p); [o]b(m);', OtherPars => 'int reflect;', PMCode => '
sub PDL::conv1d { my $opt = pop @_ if ref($_[$#_]) eq \'HASH\'; die \'Usage: conv1d( a(m), kern(p), [o]b(m), {Options} )\' if $#_<1 || $#_>2; my($a,$kern) = @_; my $c = $#_ == 2 ? $_[2] : PDL->null; &PDL::_conv1d_int($a,$kern,$c, !(defined $opt && exists $$opt{Boundary}) ? 0 : lc $$opt{Boundary} eq "reflect"); return $c; }
', Code => ' int i,i1,i2,poff; double tmp; int reflect = $COMP(reflect); int m_size = $COMP(__m_size); int p_size = $COMP(__p_size);
poff = (p_size-1)/2; for(i=0; i<m_size; i++) { tmp = 0; for(i1=0; i1<p_size; i1++) { i2 = i+i1 - poff; if (reflect && i2<0) i2 = -i2; if (reflect && i2>=m_size) i2 = m_size-(i2-m_size+1); REALMOD(i2,m_size); tmp += $a(m=>i2) * $kern(p=>i1); } $b(m=>i) = tmp; } ');
# this can be achieved by # ($a->dummy(0) == $b)->orover # but this one avoids a larger intermediate and potentially shortcuts pp_def('in', Pars => 'a(); b(n); [o] c()', Code => '$c() = 0; loop(n) %{ if ($a() == $b()) {$c() = 1; break;} %}', Doc => <<'EOD', =for ref
test if a is in the set of values b
$goodmsk = $labels->in($goodlabels); print pdl(4,3,1)->in(pdl(2,3,3)); [0 1 0]
in
is akin to the is an element of of set theory. In priciple, PDL threading could be used to achieve its functionality by using a construct like
$msk = ($labels->dummy(0) == $goodlabels)->orover;
However, in
doesn't create a (potentially large) intermediate and is generally faster. EOD );
pp_add_exported '', 'uniq'; pp_addpm << 'EOPM';
return all unique elements of a piddle
The unique elements are returned in ascending order.
print pdl(2,2,2,4,0,-1,6,6)->uniq; [-1 0 2 4 6]
Note: The returned pdl is 1D; any structure of the input piddle is lost.
See uniqind if you need the indices of the unique elements rather than the values.
EOPM
if ( $bvalflag ) { pp_addpm(<<'EOPM'); =for bad
Bad values are not considered unique by uniq and are ignored.
$a=sequence(10); $a=$a->setbadif($a%3); print $a->uniq; [0 3 6 9]
EOPM } # if: $bvalflag
pp_addpm(<<'EOPM');
return the indices of all unique elements of a piddle The order is in the order of the values to be consistent with uniq
print pdl(2,2,2,4,0,-1,6,6)->uniqind; [5, 4, 1, 3, 6]
Note: The returned pdl is 1D; any structure of the input piddle is lost.
See uniq if you want the unique values instead of the indices.
EOPM
if ($bvalflag ) { pp_addpm(<<'EOPM');
Bad values are not considered unique by uniqind and are ignored.
EOPM } # if: $bvalflag
pp_addpm(<<'EOPM');
return all unique vectors out of a collection
The unique vectors are returned in lexicographically sorted ascending order. The 0th dimension of the input PDL is treated as a dimensional index within each vector, and the 1st and any higher dimensions are taken to run across vectors. The return value is always 2D; any structure of the input PDL (beyond using the 0th dimension for vector index) is lost.
See also uniq for a uniqe list of scalars; and qsortvec for sorting a list of vectors lexicographcally.
EOPM
if ( $bvalflag ) { pp_addpm(<<'EOPM');
If a vector contains all bad values, it is ignored as in uniq. If some of the values are good, it is treated as a normal vector. For example, [1 2 BAD] and [BAD 2 3] could be returned, but [BAD BAD BAD] could not.
EOPM } # if: $bvalflag
pp_addpm(<<'EOPM');
Clip (threshold) a piddle by (optional) upper or lower bounds.
$b = $a->clip(0,3); $c = $a->clip(undef, $x);
EOD
if ( $bvalflag ) { pp_addpm(<<'EOD'); =for bad
clip handles bad values since it is just a wrapper around hclip and lclip.
EOD } # if: $bvalflag
pp_addpm(<<'EOD');
Calculate useful statistics over a dimension of a piddle
($mean,$prms,$median,$min,$max,$adev,$rms) = statsover($piddle, $weights);
This utility function calculates various useful quantities of a piddle. These are:
MEAN = sum (x)/ N
with N
being the number of elements in x
RMS = sqrt(sum( (x-mean(x))^2 )/N)
(also known as the root-mean-square deviation, or the square root of the variance)
The median is the 50th percentile data value. Median is found by medover, so WEIGHTING IS IGNORED FOR THE MEDIAN CALCULATION.
ADEV = sqrt(sum( abs(x-mean(x)) )/N)
(This is also called the standard deviation)
PRMS = sqrt( sum( (x-mean(x))^2 )/(N-1)
The population deviation is the best-estimate of the deviation of the population from which a sample is drawn.
This operator is a projection operator so the calculation will take place over the final dimension. Thus if the input is N-dimensional each returned value will be N-1 dimensional, to calculate the statistics for the entire piddle either use clump(-1)
directly on the piddle or call stats
.
', BadDoc => ' Bad values are simply ignored in the calculation, effectively reducing the sample size. If all data are bad then the output data are marked bad. ', );
pp_add_exported('','stats'); pp_addpm(<<'EOD');
Calculates useful statistics on a piddle
($mean,$prms,$median,$min,$max,$adev,$rms) = stats($piddle,[$weights]);
This utility calculates all the most useful quantities in one call. It works the same way as "statsover", except that the quantities are calculated considering the entire input PDL as a single sample, rather than as a collection of rows. See "statsover" for definitions of the returned quantities.
EOD
if ( $bvalflag ) { pp_addpm(<<'EOD'); =for bad
Bad values are handled; if all input values are bad, then all of the output values are flagged bad.
EOD } # if: bvalflag pp_addpm(<<'EOD');
Calculates a histogram$_->{Doc1} for given stepsize and minimum.
\$h = $_->{Name}(\$data, $_->{Doc2}\$step, \$min, \$numbins); \$hist = zeroes \$numbins; # Put histogram in existing piddle. $_->{Name}(\$data, $_->{Doc2}\$hist, \$step, \$min, \$numbins);
The histogram will contain \$numbins
bins starting from \$min
, each \$step
wide. The value in each bin is the $_->{Doc3}values in \$data
that lie within the bin limits.
Data below the lower limit is put in the first bin, and data above the upper limit is put in the last bin.
The output is reset in a different threadloop so that you can take a histogram of \$a(10,12)
into \$b(15)
and get the result you want. $_->{Doc4}
perldl> p $_->{Doc5}
EOD }
for( {Name => 'histogram2d', WeightPar => '', HistType => 'int+', HistOp => '++', Doc1 => "", Doc2 => "", Doc3 => "number of\n", Doc5 => "histogram2d(pdl(1,1,1,2,2),pdl(2,1,1,1,1),1,0,3,1,0,3) [ [0 0 0] [0 2 2] [0 1 0] ]
"}, {Name => 'whistogram2d', WeightPar => 'float+ wt(n);', HistType => 'float+', HistOp => '+= $wt()', Doc1 => " from weighted data", Doc2 => " \$weights,", Doc3 => "sum of the values in\n\$weights
that correspond to ", Doc5 => "whistogram2d(pdl(1,1,1,2,2),pdl(2,1,1,1,1),pdl(0.1,0.2,0.3,0.4,0.5),1,0,3,1,0,3) [ [ 0 0 0] [ 0 0.5 0.9] [ 0 0.1 0] ]
"} ) { pp_def($_->{Name}, Pars => 'ina(n); inb(n); '.$_->{WeightPar}.$_->{HistType}. '[o] hist(ma,mb)', # set outdim by Par! OtherPars => 'double stepa; double mina; int masize => ma; double stepb; double minb; int mbsize => mb;', HandleBad => 1, Code => 'register int ja,jb; register int maxja = $SIZE(ma)-1; register int maxjb = $SIZE(mb)-1; register double mina = $COMP(mina); register double minb = $COMP(minb); register double stepa = $COMP(stepa); register double stepb = $COMP(stepb); threadloop %{ loop(ma,mb) %{ $hist() = 0; %} %} threadloop %{ loop(n) %{ ja = (int) (($ina()-mina)/stepa); jb = (int) (($inb()-minb)/stepb); if (ja<0) ja=0; if (ja > maxja) ja = maxja; if (jb<0) jb=0; if (jb > maxjb) jb = maxjb; ($hist(ma => ja,mb => jb))'.$_->{HistOp}.'; %} %} ', BadCode => 'register int ja,jb; register int maxja = $SIZE(ma)-1; register int maxjb = $SIZE(mb)-1; register double mina = $COMP(mina); register double minb = $COMP(minb); register double stepa = $COMP(stepa); register double stepb = $COMP(stepb); threadloop %{ loop(ma,mb) %{ $hist() = 0; %} %} threadloop %{ loop(n) %{ if ( $ISGOOD(ina()) && $ISGOOD(inb()) ) { ja = (int) (($ina()-mina)/stepa); jb = (int) (($inb()-minb)/stepb); if (ja<0) ja=0; if (ja > maxja) ja = maxja; if (jb<0) jb=0; if (jb > maxjb) jb = maxjb; ($hist(ma => ja,mb => jb))'.$_->{HistOp}.'; } %} %} ', Doc=><<"EOD");
Calculates a 2d histogram$_->{Doc1}.
\$h = $_->{Name}(\$datax, \$datay,$_->{Doc2} \$stepx, \$minx, \$nbinx, \$stepy, \$miny, \$nbiny); \$hist = zeroes \$nbinx, \$nbiny; # Put histogram in existing piddle. $_->{Name}(\$datax, \$datay,$_->{Doc2} \$hist, \$stepx, \$minx, \$nbinx, \$stepy, \$miny, \$nbiny);
The histogram will contain \$nbinx
x \$nbiny
bins, with the lower limits of the first one at (\$minx, \$miny)
, and with bin size (\$stepx, \$stepy)
. The value in each bin is the $_->{Doc3}values in \$datax
and \$datay
that lie within the bin limits.
Data below the lower limit is put in the first bin, and data above the upper limit is put in the last bin.
perldl> p $_->{Doc5}
EOD }
########################################################### # a number of constructors: fibonacci, append, axisvalues & # random numbers ###########################################################
pp_def('fibonacci', Pars => '[o]x(n);', Doc=>'Constructor - a vector with Fibonacci\'s sequence', PMFunc=>'', PMCode=><<'EOD', sub fibonacci { ref($_[0]) && ref($_[0]) ne 'PDL::Type' ? $_[0]->fibonacci : PDL->fibonacci(@_) } sub PDL::fibonacci{ my $class = shift; my $x = scalar(@_)? $class->new_from_specification(@_) : $class->new_or_inplace; &PDL::_fibonacci_int($x->clump(-1)); return $x; } EOD Code => ' PDL_Long i=0; $GENERIC() x1, x2;
x1 = 1; x2 = 0; loop(n) %{ $x() = x1 + x2; if (i++>0) { x2 = x1; x1 = $x(); } %} ');
pp_def('append', Pars => 'a(n); b(m); [o] c(mn)', # note that ideally we want to say '$SIZE(mn) = $SIZE(m)+$SIZE(n);' # but that requires placing RedoDimsParsedCode *after* assignment of # childdims to $SIZE(XXX)!!! XXXXXmake that workXXXXX RedoDimsCode => ' pdl * dpdla = $PDL(a); pdl * dpdlb = $PDL(b); $SIZE(mn) = (dpdla->ndims > 0 ? dpdla->dims[0] : 1) + (dpdlb->ndims > 0 ? dpdlb->dims[0] : 1); ', Code => 'register PDL_Long mnp; PDL_Long ns = $SIZE(n); threadloop %{ loop(n) %{ $c(mn => n) = $a(); %} loop(m) %{ mnp = m+ns; $c(mn => mnp) = $b(); %} %}', Doc => '=for ref
append two or more piddles by concatenating along their first dimensions
$a = ones(2,4,7); $b = sequence 5; $c = $a->append($b); # size of $c is now (7,4,7) (a jumbo-piddle ;)
append
appends two piddles along their first dims. Rest of the dimensions must be compatible in the threading sense. Resulting size of first dim is the sum of the sizes of the first dims of the two argument piddles - ie n + m
.
' );
pp_addpm(<<'EOD')
$c = $a->glue(<dim>,$b,...)
Glue two or more PDLs together along an arbitrary dimension (N-D append).
Sticks $a, $b, and all following arguments together along the specified dimension. All other dimensions must be compatible in the threading sense.
Glue is permissive, in the sense that every PDL is treated as having an infinite number of trivial dimensions of order 1 -- so $a-
glue(3,$b)> works, even if $a and $b are only one dimensional.
If one of the PDLs has no elements, it is ignored. Likewise, if one of them is actually the undefined value, it is treated as if it had no elements.
If the first parameter is a defined perl scalar rather than a pdl, then it is taken as a dimension along which to glue everything else, so you can say $cube = PDL::glue(3,@image_list);
if you like.
glue
is implemented in pdl, using a combination of xchg and append. It should probably be updated (one day) to a pure PP function.
Internal routine
axisvalues
is the internal primitive that implements axisvals and alters its argument.
' ); # pp_def: axisvalues
pp_addpm(<<'EOD');
Constructor which returns piddle of random numbers
$a = random([type], $nx, $ny, $nz,...); $a = random $b;
etc (see zeroes).
This is the uniform distribution between 0 and 1 (assumedly excluding 1 itself). The arguments are the same as zeroes
(q.v.) - i.e. one can specify dimensions, types or give a template.
You can use the perl function srand to seed the random generator. For further details consult Perl's srand documentation.
Constructor which returns piddle of random numbers
$a = randsym([type], $nx, $ny, $nz,...); $a = randsym $b;
etc (see zeroes).
This is the uniform distribution between 0 and 1 (excluding both 0 and 1, cf random). The arguments are the same as zeroes
(q.v.) - i.e. one can specify dimensions, types or give a template.
You can use the perl function srand to seed the random generator. For further details consult Perl's srand documentation.
Constructor which returns piddle of Gaussian random numbers
$a = grandom([type], $nx, $ny, $nz,...); $a = grandom $b;
etc (see zeroes).
This is generated using the math library routine ndtri
.
Mean = 0, Stddev = 1
You can use the perl function srand to seed the random generator. For further details consult Perl's srand documentation.
routine for searching 1D values i.e. step-function interpolation.
$inds = vsearch($vals, $xs);
Returns for each value of $vals
the index of the least larger member of $xs
(which need to be in increasing order). If the value is larger than any member of $xs
, the index to the last element of $xs
is returned.
This function is useful e.g. when you have a list of probabilities for events and want to generate indices to events:
$a = pdl(.01,.86,.93,1); # Barnsley IFS probabilities cumulatively $b = random 20; $c = vsearch($b, $a); # Now, $c will have the appropriate distr.
It is possible to use the cumusumover function to obtain cumulative probabilities from absolute probabilities.
EOD
pp_def('interpolate', HandleBad => 0, BadDoc => 'needs major (?) work to handles bad values', Pars => 'xi(); x(n); y(n); [o] yi(); int [o] err()', GenericTypes => ['F','D'], # too restrictive ? Code => ' $GENERIC() d; long n = $SIZE(n); long n1 = n-1; int up = ($x(n => n1) > $x(n => 0)); long jl, jh, m; int carp;
threadloop %{ jl = -1; jh = n; carp = 0; while (jh-jl > 1) /* binary search */ { m = (jh+jl) >> 1; if ($xi() > $x(n => m) == up) jl = m; else jh = m; } if (jl == -1) { if ($xi() != $x(n => 0)) carp = 1; jl = 0; } else if (jh == n) { if ($xi() != $x(n => n1)) carp = 1; jl = n1-1; } jh = jl+1; if ((d = $x(n => jh)-$x(n => jl)) == 0) barf("identical abscissas"); d = ($x(n => jh)-$xi())/d; $yi() = d*$y(n => jl) + (1-d)*$y(n => jh); $err() = carp; %} ', Doc=><<'EOD');
routine for 1D linear interpolation
( $yi, $err ) = interpolate($xi, $x, $y)
Given a set of points ($x,$y)
, use linear interpolation to find the values $yi
at a set of points $xi
.
interpolate
uses a binary search to find the suspects, er..., interpolation indices and therefore abscissas (ie $x
) have to be strictly ordered (increasing or decreasing). For interpolation at lots of closely spaced abscissas an approach that uses the last index found as a start for the next search can be faster (compare Numerical Recipes hunt
routine). Feel free to implement that on top of the binary search if you like. For out of bounds values it just does a linear extrapolation and sets the corresponding element of $err
to 1, which is otherwise 0.
See also interpol, which uses the same routine, differing only in the handling of extrapolation - an error message is printed rather than returning an error piddle.
Signature: (xi(); x(n); y(n); [o] yi())
routine for 1D linear interpolation
$yi = interpol($xi, $x, $y)
interpol
uses the same search method as interpolate, hence $x
must be strictly ordered (either increasing or decreasing). The difference occurs in the handling of out-of-bounds values; here an error message is printed.
Interpolate values from an N-D piddle, with switchable method
$source = 10*xvals(10,10) + yvals(10,10); $index = pdl([[2.2,3.5],[4.1,5.0]],[[6.0,7.4],[8,9]]); print $source->interpND( $index );
InterpND acts like indexND, collapsing $index
by lookup into $source
; but it does interpolation rather than direct sampling. The interpolation method and boundary condition are switchable via an options hash.
By default, linear or sample interpolation is used, with constant value outside the boundaries of the source pdl. No dataflow occurs, because in general the output is computed rather than indexed.
All the interpolation methods treat the pixels as value-centered, so the sample
method will return $a->(0) for coordinate values on the set [-0.5,0.5), and all methods will return $a->(1) for a coordinate value of exactly 1.
Recognized options:
Values can be:
The nearest value is taken. Pixels are regarded as centered on their respective integer coordinates (no offset from the linear case).
The values are N-linearly interpolated from an N-dimensional cube of size 2.
The values are interpolated using a local cubic fit to the data. The fit is constrained to match the original data and its derivative at the data points. The second derivative of the fit is not continuous at the data points. Multidimensional datasets are interpolated by the successive-collapse method.
(Note that the constraint on the first derivative causes a small amount of ringing around sudden features such as step functions).
The source is Fourier transformed, and the interpolated values are explicitly calculated from the coefficients. The boundary condition option is ignored -- periodic boundaries are imposed.
If you pass in the option "fft", and it is a list (ARRAY) ref, then it is a stash for the magnitude and phase of the source FFT. If the list has two elements then they are taken as already computed; otherwise they are calculated and put in the stash.
This option is passed unmodified into indexND, which is used as the indexing engine for the interpolation. Some current allowed values are 'extend', 'periodic', 'truncate', and 'mirror' (default is 'truncate').
contains the fill value used for 'truncate' boundary. (default 0)
An array ref whose associated list is used to stash the FFT of the source data, for the FFT method.
Converts a one dimensional index piddle to a set of ND coordinates
@coords=one2nd($a, $indices)
returns an array of piddles containing the ND indexes corresponding to the one dimensional list indices. The indices are assumed to correspond to array $a
clumped using clump(-1)
. This routine is used in whichND, but is useful on its own occasionally.
perldl> $a=pdl [[[1,2],[-1,1]], [[0,-3],[3,2]]]; $c=$a->clump(-1) perldl> $maxind=maximum_ind($c); p $maxind; 6 perldl> print one2nd($a, maximum_ind($c)) 0 1 1 perldl> p $a->at(0,1,1) 3
Returns indices of non-zero values from a 1-D PDL
$i = which($mask);
returns a pdl with indices for all those elements that are nonzero in the mask. Note that the returned indices will be 1D. If you feed in a multidimensional mask, it will be flattened before the indices are calculated. See also whichND for multidimensional masks.
If you want to index into the original mask or a similar piddle with output from which
, remember to flatten it before calling index:
$data = random 5, 5; $idx = which $data > 0.5; # $idx is now 1D $bigsum = $data->flat->index($idx)->sum; # flatten before indexing
Compare also where for similar functionality.
SEE ALSO:
which_both returns separately the indices of both zero and nonzero values in the mask.
where returns associated values from a data PDL, rather than indices into the mask PDL.
whichND returns N-D indices into a multidimensional PDL.
perldl> $x = sequence(10); p $x [0 1 2 3 4 5 6 7 8 9] perldl> $indx = which($x>6); p $indx [7 8 9]
EOD
my $doc_which_both = <<'EOD';
Returns indices of zero and nonzero values in a mask PDL
($i, $c_i) = which_both($mask);
This works just as which, but the complement of $i
will be in $c_i
.
perldl> $x = sequence(10); p $x [0 1 2 3 4 5 6 7 8 9] perldl> ($small, $big) = which_both ($x >= 5); p "$small\n $big" [5 6 7 8 9] [0 1 2 3 4]
EOD
for ( {Name=>'which', Pars => 'mask(n); int [o] inds(m);', Variables => 'int dm=0;', Elseclause => "", Autosize => '$SIZE(m) = sum;', Doc => $doc_which, PMCode=><<'EOD', sub which { my ($this,$out) = @_; $this = $this->flat; $out = $this->nullcreate unless defined $out; PDL::_which_int($this,$out); return $out; } *PDL::which = \&which; EOD }, {Name => 'which_both', Pars => 'mask(n); int [o] inds(m); int [o]notinds(q)', Variables => 'int dm=0; int dm2=0;', Elseclause => "else { \n \$notinds(q => dm2)=n; \n dm2++;\n }", Autosize => '$SIZE(m) = sum;'."\n".' $SIZE(q) = dpdl->dims[0]-sum;', Doc => $doc_which_both, PMCode=><<'EOD', sub which_both { my ($this,$outi,$outni) = @_; $this = $this->flat; $outi = $this->nullcreate unless defined $outi; $outni = $this->nullcreate unless defined $outni; PDL::_which_both_int($this,$outi,$outni); return wantarray ? ($outi,$outni) : $outi; } *PDL::which_both = \&which_both; EOD } ) { pp_def($_->{Name}, HandleBad => 1, Doc => $_->{Doc}, Pars => $_->{Pars}, PMCode => $_->{PMCode}, Code => $_->{Variables} . 'loop(n) %{ if($mask()) { $inds(m => dm) = n; dm++; }'.$_->{Elseclause} . "\n". ' %}', BadCode => $_->{Variables} . 'loop(n) %{ if ( $mask() && $ISGOOD($mask()) ) { $inds(m => dm) = n; dm++; }'.$_->{Elseclause} . "\n". ' %}',
# the next one is currently a dirty hack # this will probably break once dataflow is enabled again # *unless* we have made sure that mask is physical by now!!! RedoDimsCode => ' PDL_Long sum = 0; /* not sure if this is necessary */ pdl * dpdl = $PDL(mask); $GENERIC() *m_datap = (($GENERIC() *)(PDL_REPRP(dpdl))); PDL_Long inc = PDL_REPRINC(dpdl,0); PDL_Long offs = PDL_REPROFFS(dpdl); int i;
if (dpdl->ndims != 1) barf("dimflag currently works only with 1D pdls");
'. ($bvalflag ? ' if(dpdl->state & PDL_BADVAL) for (i=0; i<dpdl->dims[0]; i++) { $GENERIC() foo = *(m_datap+inc*i+offs); if(foo && $ISGOODVAR(foo,mask) )sum++; } else ':'').' for (i=0; i<dpdl->dims[0]; i++) { $GENERIC() foo = *(m_datap+inc*i+offs); if(foo) sum++; }
'. $_->{Autosize} . ' /* printf("RedoDimsCode: setting dim m to %ld\n",sum); */' ); }
pp_addpm(<<'EOD' =head2 where
Use a mask to select values from one or more data PDLs
where
accepts one or more data piddles and a mask piddle. It returns a list of output piddles, corresponding to the input data piddles. Each output piddle is a 1-dimensional list of values in its corresponding data piddle. The values are drawn from locations where the mask is nonzero.
The output PDLs are still connected to the original data PDLs, for the purpose of dataflow.
where
combines the functionality of which and index into a single operation.
BUGS:
There is no whereND
, and probably should be. While where
works OK for most N-dimensional cases, it does not thread properly over (for example) the (N+1)th dimension in data that is compared to an N-dimensional mask.
$i = $x->where($x+5 > 0); # $i contains those elements of $x # where mask ($x+5 > 0) is 1 $i .= -5; # Set those elements (of $x) to -5. Together, these # commands clamp $x to a maximum of -5.
It is also possible to use the same mask for several piddles with the same call:
($i,$j,$k) = where($x,$y,$z, $x+5>0);
Note: $i
is always 1-D, even if $x
is >1-D.
WARNING: The first argument (the values) and the second argument (the mask) currently have to have the exact same dimensions (or horrible things happen). You *cannot* thread over a smaller mask, for example.
Return the coordinates of non-zero values in a mask.
WhichND returns the N-dimensional coordinates of each nonzero value in a mask PDL with any number of dimensions.
For historical reasons the return value is different in list and scalar context. In scalar context, you get back a PDL containing coordinates suitable for use in indexND or range; in list context, the coordinates are broken out into separate PDLs.
$coords = whichND($mask);
returns a PDL containing the coordinates of the elements that are non-zero in $mask
, suitable for use in indexND. The 0th dimension contains the full coordinate listing of each point; the 1st dimension lists all the points. For example, if $mask has rank 4 and 100 matching elements, then $coords has dimension 4x100.
@coords=whichND($mask);
returns a perl list of piddles containing the coordinates of the elements that are non-zero in $mask
. Each element corresponds to a particular index dimension. For example, if $mask has rank 4 and 100 matching elements, then @coords has 4 elements, each of which is a pdl of size 100.
SEE ALSO:
which finds coordinates of nonzero values in a 1-D mask.
where extracts values from a data PDL that are associated with nonzero values in a mask PDL.
perldl> $a=sequence(10,10,3,4) perldl> ($x, $y, $z, $w)=whichND($a == 203); p $x, $y, $z, $w [3] [0] [2] [0] perldl> print $a->at(list(cat($x,$y,$z,$w))) 203
Implements simple set operations like union and intersection
Usage: $set = setops($a, <OPERATOR>, $b);
The operator can be OR
, XOR
or AND
. This is then applied to $a
viewed as a set and $b
viewed as a set. The functioning is as follows:
OR
The resulting vector will contain the elements that are either in $a
or in $b
or both. This is the union in set operation terms
XOR
The resulting vector will contain the elements that are either in $a
or $b
, but not in both. This is
Union($a, $b) - Intersection($a, $b)
in set operation terms.
AND
The resulting vector will contain the intersection of $a
and $b
, so the elements that are in both $a
and $b
. Note that for convenience this operation is also aliased to intersect
It should be emphasized that these routines are used when one or both of the sets $a
, $b
are hard to calculate or that you get from a separate subroutine.
Finally IDL users might be familiar with Craig Markwardt's cmset_op.pro
routine which has inspired this routine although it was written independently However the present routine has a few less options (but see the exampels)
You will very often use these functions on an index vector, so that is what we will show here. We will in fact something slightly silly. First we will find all squares that are also cubes below 10000.
Create a sequence vector:
perldl> $x = sequence(10000)
Find all odd and even elements:
perldl> ($even, $odd) = which_both( ($x % 2) == 0)
Find all squares
perldl> $squares= which(ceil(sqrt($x)) == floor(sqrt($x)))
Find all cubes (being careful with roundoff error!)
perldl> $cubes= which(ceil($x**(1.0/3.0)) == floor($x**(1.0/3.0)+1e-6))
Then find all squares that are cubes:
perldl> $both = setops($squares, 'AND', $cubes)
And print these (assumes that PDL::NiceSlice
is loaded!)
perldl> p $x($both) [0 1 64 729 4096]
Then find all numbers that are either cubes or squares, but not both:
perldl> $cube_xor_square = setops($squares, 'XOR', $cubes) perldl> p $cube_xor_square->nelem() 112
So there are a total of 112 of these!
Finally find all odd squares:
perldl> $odd_squares = setops($squares, 'AND', $odd)
Another common occurance is to want to get all objects that are in $a
and in the complement of $b
. But it is almost always best to create the complement explicitly since the universe that both are taken from is not known. Thus use which_both if possible to keep track of complements.
If this is impossible the best approach is to make a temporary:
This creates an index vector the size of the universe of the sets and set all elements in $b
to 0
perldl> $tmp = ones($n_universe); $tmp($b)=0;
This then finds the complement of $b
perldl> $C_b = which($tmp == 1);
and this does the final selection:
perldl> $set = setops($a, 'AND', $C_b)
Calculate the intersection of two piddles
Usage: $set = intersect($a, $b);
This routine is merely a simple interface to setops. See that for more information
Find all numbers less that 100 that are of the form 2*y and 3*x
perldl> $x=sequence(100)
perldl> $factor2 = which( ($x % 2) == 0)
perldl> $factor3 = which( ($x % 3) == 0)
perldl> $ii=intersect($factor2, $factor3)
perldl> p $x($ii)
[0 6 12 18 24 30 36 42 48 54 60 66 72 78 84 90 96]