Statistics::Data - Load, access, update, check and save one or more sequences of data for statistical analysis
This is documentation for Version 0.09 of Statistics/Data.pm, released February 2015.
use Statistics::Data 0.09; my $dat = Statistics::Data->new(); # managing labelled sequences: $dat->load({'aname' => \@data1, 'anothername' => \@data2}); # labels are arbitrary $aref = $dat->access(label => 'aname'); # gets back a copy of @data1 $dat->add(aname => [2, 3]); # pushes new values onto loaded copy of @data1 $dat->dump_list(); # print to check if both arrays are loaded and their number of elements $dat->unload(label => 'anothername'); # only 'aname' data remains loaded $aref = $dat->access(label => 'aname'); # $aref is a reference to a copy of @data1 $dat->dump_vals(label => 'aname', delim => ','); # proof in print it's back # managing multiple anonymous sequences: $dat->load(\@data1, \@data2); # any number of anonymous arrays $dat->add([2], [6]); # pushes a single value apiece onto copies of @data1 and @data2 $aref = $dat->access(index => 1); # returns reference to copy of @data2, with its new values $dat->unload(index => 0); # only @data2 remains loaded, and its index is now 0 # managing single anonymous sequence, over time: $dat->load(1, 2, 2); $dat->add(1); # loaded sequence is now 1, 2, 2, 1 $dat->dump_vals(); # same as: print @{$dat->access()}, "\n"; $dat->save_to_file(path => 'variables1.dat'); # write to disk $dat->unload(); # all gone - go wild, knowing that you can ... $dat->load_from_path(path => 'variables1.dat'); # back again - go to work $dat->dump_vals(); # proof: same printed output as before # do more work
Handles data for some other statistics modules, as in loading, updating and retrieving data for analysis. Performs no actual statistical analysis itself.
Rationale is not wanting to write the same or similar load, add, etc. methods for every statistics module, not to provide an omnibus API for Perl stat modules. It, however, encompasses much of the variety of how Perl stats modules do the basic handling their data. Used for Statistics::Sequences (and its sub-tests).
Manages caches of one or more lists of data for use by some other statistics modules. The lists are ordered arrays comprised of literal scalars (numbers, strings). They can be loaded, added to (updated), accessed or unloaded by referring to the index (order) in which they have been loaded (or previously added to), or by a particular label. The lists are cached within the class object's '_DATA' aref as an aref itself, optionally associated with a 'label'. The particular structures supported here to load, update, retrieve, unload data are specified under load. Any module that uses this one as its base can still use its own rules to select the appropriate sequence, or provide the appropriate sequence within the call to itself.
$dat = Statistics::Data->new();
Returns a new Statistics::Data object.
$new_self = $dat->clone();
Alias: clone
Returns a copy of the class object with its data loaded (if any). Note this is not a copy of any particular data but the whole blessed hash. Alternatively, use pass to get all the data added to a new object, or use access to load/add particular arrays of data into another object. Nothing modified in this new object affects the original.
Methods to cache and uncache data into the data-object.
$dat->load(@data); # CASE 1 - can be updated/retrieved anonymously, or as index => i (load order) $dat->load(\@data); # CASE 2 - same, as aref $dat->load(data => \@data); # CASE 3 - updated/retrieved as label => 'data' (arbitrary name, not just 'data'); or by index (order) $dat->load({ data => \@data }) # CASE 4 - same as CASE 4, as hashref $dat->load(blues => \@blue_data, reds => \@red_data); # CASE 5 - same as CASE 3 but with multiple named loads $dat->load({ blues => \@blue_data, reds => \@red_data }); # CASE 6 - same as CASE 5 bu as hashref $dat->load(\@blue_data, \@red_data); # CASE 7 - same as CASE 2 but with multiple aref loads # Not supported: #$dat->load(data => @data); # not OK - use CASE 3 instead #$dat->load([\@blue_data, \@red_data]); # not OK - use CASE 7 instead #$dat->load([ [blues => \@blue_data], [reds => \@red_data] ]); # not OK - use CASE 5 or CASE 6 instead #$dat->load(blues => \@blue_data, reds => [\@red_data1, \@red_data2]); # not OK - too mixed to make sense
Alias: load_data
Cache a list of data as an array-reference. Each call removes previous loads, as does sending nothing. If data need to be cached without unloading previous loads, try add. Arguments with the following structures are acceptable as data, and will be accessible by either index or label as expected:
Load an anonymous array that has no named values. For example:
$dat->load(1, 4, 7); $dat->load(@ari);
This is loaded as a single sequence, with an undefined label, and indexed as 0. Note that trying to load a labelled dataset with an unreferenced array is wrong for it will be treated like this case - the label will be "folded" into the sequence itself.
Load a reference to a single anonymous array that has no named values, e.g.:
$dat->load([1, 4, 7]); $dat->load(\@ari);
This is loaded as a single sequence, with an undefined label, and indexed as 0.
Same as above, but note that more than one unlabelled array-reference can also be loaded at once, e.g.:
$dat->load([1, 4, 7], [2, 5, 9]); $dat->load(\@ari1, \@ari2);
Each sequence can be accessed, using access, by specifying index => index, the latter value representing the order in which these arrays were loaded.
Load one or more labelled references to arrays, e.g.:
$dat->load('dist1' => [1, 4, 7]); $dat->load('dist1' => [1, 4, 7], 'dist2' => [2, 5, 9]);
This loads the sequence(s) with a label attribute, so that when calling access, they can be retrieved by name, e.g., passing label => 'dist1'. The load method involves a check that there is an even number of arguments, and that, if this really is a hash, all the keys are defined and not empty, and all the values are in fact array-references.
As above, but where the hash is referenced, e.g.:
$dat->load({'dist1' => [1, 4, 7], 'dist2' => [2, 5, 9]});
This means that using the following forms will produce unexpected results, if they do not actually croak, and so should not be used:
$dat->load(data => @data); # no croak but wrong - puts "data" in @data - use \@data $dat->load([\@blue_data, \@red_data]); # use unreferenced ARRAY of AREFs instead $dat->load([ [blues => \@blue_data], [reds => \@red_data] ]); # treated as single AREF; use HASH of AREFs instead $dat->load(blues => \@blue_data, reds => [\@red_data1, \@red_data2]); # mixed structures not supported
Alias: add_data, append_data, update
Same usage as above for load. Just push any value(s) or so along, or loads an entirely labelled sequence, without clobbering what's already in there (as load would). If data have not been loaded with a label, then appending data to them happens according to the order of array-refs set here, see EXAMPLES could even skip adding something to one previously loaded sequence by, e.g., going $dat->add([], \new_data) - adding nothing to the first loaded sequence, and initialising a second array, if none already, or appending these data to it.
$dat->unload(); # deletes all cached data, named or not $dat->unload(index => integer); # deletes the aref named 'data' whatever $dat->unload(label => 'a name'); # deletes the aref named 'data' whatever
Empty, clear, clobber what's in there. Croaks if given index or label does not refer to any loaded data. This should be used whenever any already loaded or added data are no longer required ahead of another add, including via copy or share.
$dat_new->share($dat_old);
Adds all the data from one Statistics::Data object to another. Changes in the new copies do not affect the originals.
$dat->load_from_file(path => 'medata.csv', format => 'xml|csv'); $dat->load_from_file(path => 'mysequences.csv', serializer => 'XML::Simple', compress => 1, secret => '123'); # serialization options
Loads data from a file, assuming there are data in the given path that have been saved in the format used in save_to_file. Basically a wrapper to access method in Data::Serializer; cf. for options; and then to load. If the data retreived are actually to be added to any data already cached via a previous load or add, define the optional parameter keep => 1.
To retrieve what has been previously loaded, simply call access, specifying the "label" or "index" that was used to load/add the data - i.e., when loaded as a hashref or an arrayref, respectively; specifying the list by label (as loaded hash-wise) or index (as loaded array-wise).
For retrieving more than one previously loaded dataset, use one of the "get" methods, choosing between getting back a hash- or an array-ref, or to get back a single list, as by access, after all. These "get" methods only support retrieving data loaded as hashrefs for now; use access to get back index-specific loads. They might be folded within access later on.
$aref = $dat->access(); #returns the first and/or only sequence loaded, if any $aref = $dat->access(index => integer); #returns the ith sequence loaded $aref = $dat->access(label => 'a_name'); # returns a particular named cache of data
Alias: get_data
Returns one referenced array being previously loaded/added to data by the given index (in a flat-list load) or label (in a hash-wise load). Same as calling get_aref_by_lab.
$href = $data->get_hoa(lab => [qw/fez boa/]); # retrieve 1 or more named data $href = $data->get_hoa(); # retrieve all named data
Returns a hashref of arefs, where the keys are the names of the data, as previously given in a load, and the values are arefs of the list of data that has been loaded for that name.
The optional argument lab should be a reference to a list of one or more data that have been given as keys in a hash-wise load. Any elements in this list that have not been used as names in a load are ignored. If none of the names has been used, an empty list is returned. If there is no lab argument, then all of the loaded data are returned as a hashref of arefs; if there were no named data, this a reference to an empty hash.
This is useful in a module like Statistics::ANOVA::JT that needs to continuously cross-refer to multiple variables to make a single calculation while also being able to distinguish them by some meaningful key other than simply an index number.
For working with numerical data in particular, see the following two methods.
$hoa = $dat->get_hoa_by_lab_numonly_indep(); # same as get_hoa but each list culled of NaNs
Returns the variables given in the argument lab (an aref of strings) culled on any empty or non-numeric values. This is done by treating each variable indpendently, with culls on one "list" not creating a cull on any other. This is the type of data useful for an independent ANOVA.
$hoa = $dat->get_hoa_by_lab_numonly_across(); # same as get_hoa but each list culled of NaNs at same i across lists
Returns hashref of previously loaded variable data (as arefs) culled of an empty or non-numerical values whereby even a valid value in one list is culled if it is at an index that is invalid in another list. This is the type of data useful for a dependent ANOVA.
$aref_of_arefs = $dat->get_aoa_by_lab();
Returns a reference to an array where each value is itself an array of data, as separately loaded under a different name or anonymously, in the order that they were loaded. If no lab value is defined, all the loaded data are returned as a list of arefs.
$aref = $dat->get_aref_by_lab();
Returns a reference to a single, previously loaded hashref of arrayed of data, as specified in the named argument lab. The array is empty if no data have been loaded, or if there is none with the given lab. If lab is not defined, the the last-loaded data, if any, is returned (as aref).
$n = $dat->ndata();
Returns the number of loaded variables.
$aref = $dat->labels();
Returns a reference to an array of all the datanames (labels), if any.
$bool = $dat->all_full(\@data); # test data are valid before loading them $bool = $dat->all_full(label => 'mydata'); # checking after loading/adding the data (or key in 'index')
Checks not only if the data sequence, as named or indexed, exists, but if it is non-empty: has no empty elements, with any elements that might exist in there being checked with hascontent.
$bool = $dat->all_numeric(); # test data first-loaded, if any $bool = $dat->all_numeric(\@data); # test these data are valid before loading them $bool = $dat->all_numeric(label => 'mydata'); # check specific data after loading/adding them by a 'label' or by their 'index' order ($aref, $bool) = $dat->all_numeric([3, '', 4.7, undef, 'b']); # returns ([3, 4.7], 0); - same for any loaded data
Given an aref of data, or reference to data previously loaded (see access), tests numeracy of each element, and return, if called in scalar context, a boolean scalar indicating if all data in this aref are defined and not empty (using nocontent in "nocontent" in String::Util::String::Util), and, if they have content, if these are all numerical, using looks_like_number in Scalar::Util. Alternatively, if called in array context, returns the data (as an aref) less any values that failed this test, followed by the boolean.
nocontent
looks_like_number
$bool = $dat->all_proportions(\@data); # test data are valid before loading them $bool = $dat->all_proportions(label => 'mydata'); # checking after loading/adding the data (or key in 'index')
Ensure data are all proportions. Sometimes, the data a module needs are all proportions, ranging from 0 to 1 inclusive. A dataset might have to be cleaned
$bool = $dat->all_counts(\@data); # test data are valid before loading them $bool = $dat->all_counts(label => 'mydata'); # checking after loading/adding the data (or key in 'index') ($aref, $bool) = $dat->all_counts(\@data);
Returns true if all values in given data are real positive integers or zero, as well as satisfying "hascontent" and "looks_like_number" methods; false otherwise. Called in array context, returns aref of data culled of any values that are false on this basis, and then the boolean. For example, [2.2, 3, 4] and [-1, 3, 4] both fail, but [1, 3, 4] is true. Integer test is simply if $v == int($v).
$bool = $dat->all_pos(\@data); # test data are valid before loading them $bool = $dat->all_pos(label => 'mydata'); # checking after loading/adding the data (or key in 'index') ($aref, $bool) = $dat->all_pos(\@data);
Returns true if all values in given data are greater than zero, as well as "hascontent" and "looks_like_number"; false otherwise. Called in array context, returns aref of data culled of any values that are false on this basis, and then the boolean.
If the given data or aref of variable names all have the same number of elements, then that number is returned; otherwise 0.
Given an aref, returns a reference to an array of indices for a particular dataset (list) where the values are either undefined, empty or non-numerical.
$seq->dump_vals(delim => ", "); # assumes the first (only?) loaded sequence should be dumped $seq->dump_vals(index => I<int>, delim => ", "); # dump the i'th loaded sequence $seq->dump_vals(label => 'mysequence', delim => ", "); # dump the sequence loaded/added with the given "label"
Prints to STDOUT a space-separated line (ending with "\n") of a loaded/added data's elements. Optionally, give a value for delim to specify how the elements in each sequence should be separated; default is a single space.
Dumps a list (using Text::SimpleTable) of the data currently loaded, without showing their actual elements. List is firstly by index, then by label (if any), then gives the number of elements in the associated sequence.
$dat->save_to_file(path => 'mysequences.csv'); $dat->save_to_file(path => 'mysequences.csv', serializer => 'XML::Simple', compress => 1, secret => '123'); # serialization options
Saves the data presently loaded in the Statistics::Data object to a file, with the given path. This can be retrieved, with all the data added to the Statistics::Data object, via load_from_file. Basically a wrapper to store method in Data::Serializer; cf. for options.
store
1. Multivariate data (a tale of horny frogs)
In a study of how doing mental arithmetic affects arousal in self and others (i.e., how mind, body and world interact), three male frogs were maths-trained and then, as they did their calculations, were measured for pupillary dilation and perceived attractiveness. After four runs, average measures per frog can be loaded:
$frogs->load(Names => [qw/Freddo Kermit Larry/], Pupil => [59.2, 77.7, 56.1], Attract => [3.11, 8.79, 6.99]);
But one more frog still had to graudate from training, and data are now ready for loading:
$frogs->add(Names => ['Sleepy'], Pupil => [83.4], Attract => [5.30]); $frogs->dump_data(label => 'Pupil'); # prints "59.2 77.7 56.1 83.4" : all 4 frogs' pupil data for analysis by some module
Say we're finished testing for now, so:
$frogs->save_to_file(path => 'frogs.csv'); $frogs->unload();
But another frog has been trained, measures taken:
$frogs->load_from_file(path => 'frogs.csv'); $frogs->add(Pupil => [93], Attract => [6.47], Names => ['Jack']); # add yet another frog's data $frogs->dump_data(label => 'Pupil'); # prints "59.2 77.7 56.1 83.4 93": all 5 frogs' pupil data
Now we run another experiment, taking measures of heart-rate, and can add them to the current load of data for analysis:
$frogs->add(Heartrate => [.70, .50, .44, .67, .66]); # add entire new sequence for all frogs print "heartrate data are bung" if ! $frogs->all_proportions(label => 'Heartrate'); # validity check (could do before add) $frogs->dump_list(); # see all four data-sequences now loaded, each with 5 observations (1 per frog), i.e.: .-------+-----------+----. | index | label | N | +-------+-----------+----+ | 0 | Names | 5 | | 1 | Attract | 5 | | 2 | Pupil | 5 | | 3 | Heartrate | 5 | '-------+-----------+----'
2. Using as a base module
As Statistics::Sequences, and so its sub-modules, use this module as their base, it doesn't have to do much data-managing itself:
use Statistics::Sequences; my $seq = Statistics::Sequences->new(); $seq->load(qw/f b f b b/); # using Statistics::Data method say $seq->p_value(stat => 'runs', exact => 1); # using Statistics::Sequences::Runs method
Or if these data were loaded directly within Statistics::Data, the data can be shared around modules that use it as a base:
use Statistics::Data; use Statistics::Sequences::Runs; my $dat = Statistics::Data->new(); my $runs = Statistics::Sequences::Runs->new(); $dat->load(qw/f b f b b/); $runs->pass($dat); say $runs->p_value(exact => 1);
Croaked when attempting to load or add data with an unsupported data structure where the first argument is a reference. See the examples under load for valid (and invalid) ways of sending data to them.
Croaked when calling access, or any methods that use it internally -- viz., dump_vals and the validity checks all_numeric -- when it is called with a label for data that have not been loaded, or did not load successfully.
Croaked when calling unload with an index or a label attribute and the data these refer to have not been loaded, or did not load successfully.
Croaked when calling save_to_file or load_from_file without a value for the required path argument, or if it does not exist when it's touched for a load.
List::AllUtils - used for its all method when testing loads
all
Number::Misc - used for its is_even method when testing loads
is_even
String::Util - used for its hascontent and nocontent methods
hascontent
Data::Serializer - required for save_to_file and load_from_file
Scalar::Util - required for all_numeric
Text::SimpleTable - required for dump_list
Some methods rely on accessing previously loaded data but should permit performing their operations on data submitted directly to them, just like, e.g., $dat->all_numeric(\@data) is ok. This is handled for now internally, but should be handled in the same way by modules using this one as its base - for at the moment they have to check for an aref to their data-manipulating methods ahead of accessing any loaded data by this module.
Please report any bugs or feature requests to bug-statistics-data-0.01 at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Statistics-Data-0.01. This will notify the author, and then you'll automatically be notified of progress on your bug as any changes are made.
bug-statistics-data-0.01 at rt.cpan.org
You can find documentation for this module with the perldoc command.
perldoc Statistics::Data
You can also look for information at:
RT: CPAN's request tracker
http://rt.cpan.org/NoAuth/Bugs.html?Dist=Statistics-Data-0.09
AnnoCPAN: Annotated CPAN documentation
http://annocpan.org/dist/Statistics-Data-0.09
CPAN Ratings
http://cpanratings.perl.org/d/Statistics-Data-0.09
Search CPAN
http://search.cpan.org/dist/Statistics-Data-0.09/
Roderick Garton, <rgarton at cpan.org>
<rgarton at cpan.org>
Copyright 2009-2015 Roderick Garton
This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License. See perl.org for more information.
To install Statistics::Data, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Statistics::Data
CPAN shell
perl -MCPAN -e shell install Statistics::Data
For more information on module installation, please visit the detailed CPAN module installation guide.