Config::Model::Tester - Test framework for Config::Model
version 3.006
In your test file (typically t/model_test.t):
t/model_test.t
use warnings; use strict; use Config::Model::Tester ; use ExtUtils::testlib; run_tests() ;
Run tests with:
perl t/model_test.t [ --log ] [--error] [--trace] [ subtest [ test_case ] ]
This class provides a way to test configuration models with tests files. This class was designed to tests several models and several tests cases per model.
A specific layout for test files must be followed.
Each subtest is defined in a file like:
t/model_tests.d/<app-name>-test-conf.pl
This file specifies that app-name (which is defined in lib/Config/Model/*.d directory) will be used for the test cases defined in the *-test-conf.pl file.
app-name
lib/Config/Model/*.d
*-test-conf.pl
This file contains a list of test case (explained below) and expects a set of files used as test data. The layout of these test data files is explained in next section.
Each test case is represented by a configuration file (not a directory) in the *-examples directory. This configuration file will be used by the model to test and is copied as $confdir/$conf_file_name using the global variables explained below.
*-examples
$confdir/$conf_file_name
In the example below, we have 1 app model to test: lcdproc and 2 tests cases. The app name matches the file specified in lib/Config/Model/*.d directory. In this case, the app name matches lib/Config/Model/system.d/lcdproc
lcdproc
lib/Config/Model/system.d/lcdproc
t |-- model_test.t \-- model_tests.d # do not change directory name |-- lcdproc-test-conf.pl # subtest specification for lcdproc app \-- lcdproc-examples |-- t0 # test case t0 \-- LCDD-0.5.5 # test case for older LCDproc
Subtest specification is written in lcdproc-test-conf.pl file (i.e. this modules looks for files named like <app-name>-test-conf.pl>).
lcdproc-test-conf.pl
<app-name>-test-conf.pl>
Subtests data is provided in files in directory lcdproc-examples ( i.e. this modules looks for test data in directory <model-name>-examples>. lcdproc-test-conf.pl contains instructions so that each file will be used as a /etc/LCDd.conf file during each test case.
lcdproc-examples
<model-name>-examples>
/etc/LCDd.conf
lcdproc-test-conf.pl can contain specifications for more test cases. Each test case requires a new file in lcdproc-examples directory.
See "Examples" for a link to the actual LCDproc model tests
When a configuration is spread over several files, each test case is provided in a sub-directory. This sub-directory is copied in $conf_dir (a global variable as explained below)
$conf_dir
In the example below, the test specification is written in dpkg-test-conf.pl. Dpkg layout requires several files per test case. dpkg-test-conf.pl will contain instructions so that each directory under dpkg-examples will be used.
dpkg-test-conf.pl
dpkg-examples
t/model_tests.d \-- dpkg-test-conf.pl # subtest specification \-- dpkg-examples \-- libversion # example subdir, used as test case name \-- debian # directory for used by test case |-- changelog |-- compat |-- control |-- copyright |-- rules |-- source | \-- format \-- watch
See "Examples" for a link to the (many) Dpkg model tests
Each test case is a sub-directory on the *-examples directory and contains several files. The destination of the test files may depend on the system (e.g. the OS). For instance, system wide ssh_config is stored in /etc/ssh on Linux, and directly in /etc on MacOS.
ssh_config
/etc/ssh
/etc
These files are copied in a test directory using a setup parameter:
setup
setup => { test_file_in_example_dir => 'destination' }
Let's consider this example of 2 tests cases for ssh:
t/model_tests.d/ |-- ssh-test-conf.pl |-- ssh-examples \-- basic |-- system_ssh_config \-- user_ssh_config
Unfortunately, user_ssh_config is a user file, so you specify where the home directory for the tests with another global variable:
user_ssh_config
$home_for_test = '/home/joe' ;
For Linux only, the setup parameter is:
setup => { 'system_ssh_config' => '/etc/ssh/ssh_config', 'user_ssh_config' => "~/.ssh/config" }
On the other hand, system wide config file is different on MacOS and the test file must be copied in the correct location. When the value of the setup hash is another hash, the key of this other hash is used as to specify the target location for other OS (as returned by Perl $^O variable:
$^O
setup => { 'system_ssh_config' => { 'darwin' => '/etc/ssh_config', 'default' => '/etc/ssh/ssh_config', }, 'user_ssh_config' => "~/.ssh/config" }
See the actual Ssh and Sshd model tests
Each model subtest is specified in <model>-test-conf.pl. This file contains a set of global variables. (yes, global variables are often bad ideas in programs, but they are handy for tests):
<model>-test-conf.pl
# config file name (used to copy test case into test wr_root/model_tests directory) $conf_file_name = "fstab" ; # config dir where to copy the file (optional) #$conf_dir = "etc" ; # home directory for this test $home_for_test = '/home/joe' ;
Here, t0 file will be copied in wr_root/model_tests/test-t0/etc/fstab.
t0
wr_root/model_tests/test-t0/etc/fstab
# config model name to test $model_to_test = "Fstab" ; # list of tests. This modules looks for @tests global variable @tests = ( { # test name name => 't0', # add optional specification here for t0 test }, { name => 't1', # add optional specification here for t1 test }, ); 1; # to keep Perl happy
You can suppress warnings by specifying no_warnings => 1. On the other hand, you may also want to check for warnings specified to your model. In this case, you should avoid specifying no_warnings here and specify warning tests or warning filters as mentioned below.
no_warnings => 1
no_warnings
See actual fstab test.
A test file can be skipped using $skip global variable.
$skip
In this example, test is skipped when not running on a Debian system:
eval { require AptPkg::Config; }; $skip = ( $@ or not -r '/etc/debian_version' ) ? 1 : 0;
Some tests will require the creation of a configuration class dedicated for test (typically to test corner cases on a backend).
This test class can be created directly in the test specification by calling create_config_class on $model variable. See for instance the layer test or the test for shellvar backend.
$model
In some models like Multistrap, the config file is chosen by the user. In this case, the file name must be specified for each tests case:
Multistrap
# not needed if test file is named multistrap-test-conf.pl $model_to_test = "Multistrap"; @tests = ( { name => 'arm', config_file => '/home/foo/my_arm.conf', check => {}, }, );
See the actual multistrap test.
Some application like systemd requires a backend argument specified by User (e.g. a service name for systemd). The parameter backend_arg can be specified to emulate this behavior.
backend_arg
When the input data for test is quite complex (several files), it may be interested to re-use these data for other tests case. Knowing that test name must must unique, you can re-use test data with data_from parameter. For instance:
data_from
@tests = ( { name => 'some-test', # ... }, { name => 'some-other-test', data_from => 'some-test', # re-use data from test above # ... },
See plainfile backend test for a real life example.
Each subtest follow a sequence explained below. Each step of this sequence may be altered by adding specification in <model-to-test>-test-conf.pl:
<model-to-test>-test-conf.pl
Setup test in wr_root/model_tests/<subtest name>/. If your configuration file layout depend on the target system, you will have to specify the path using setup parameter. See "Test file layout depending on system".
wr_root/model_tests/<subtest name>/
Create configuration instance, load config data and check its validity. Use load_check => 'no' if your file is not valid.
load_check => 'no'
Check for config data warnings. You should pass the list of expected warnings that are emitted through Log::Log4Perl. The array ref is passed as is to the expect function of "expect" in Test::Log::Lo4Perl. E.g:
expect
log4perl_load_warnings => [ [ 'Tree.Node', (warn => qr/deprecated/) x 2 ] , [ 'Tree.Element.Value' , ( warn => qr/skipping/) x 2 ] ]
The Log classes are specified in cme/Logging.
cme/Logging
Log levels below "warn" are ignored.
Config::Model is currently transitioning from traditional "warn" to warn logs. To avoid breaking all tests based on this module, the warnings are emitted through Log::Log4Perl only when c<$::_use_log4perl_to_warn> is set. This hack will be removed once all warnings checks in tests are ported to log4perl checks.
DEPRECATED. Check for config data warning. You should pass the list of expected warnings. E.g.
load_warnings => [ qr/Missing/, (qr/deprecated/) x 3 , ],
Use an empty array_ref to mask load warnings.
Optionally run update command:
update => { [ returns => 'foo' , ] no_warnings => [ 0 | 1 ], # default 0 quiet => [ 0 | 1], # default 0, passed to update method loag4perl_update_warnings => [ ... ] # Test::Log::Log4Perl::expect arguments }
Where:
returns is the expected return value (optional).
returns
no_warnings to suppress the warnings coming from Config::Model::Value. Note that no_warnings => 1 may be useful for less verbose test.
quiet to suppress progress messages during update.
quiet
log4perl_update_warnings is used to check the warnings produced during update. The argument is passed to expect function of Test::Log::Log4Perl. See load_warnings parameter above for more details.
log4perl_update_warnings
load_warnings
DEPRECATED. update_warnings is an array ref of quoted regexp (See qr operator) to check the warnings produced during update. use update => [] to check that no warnings are issued during update.
update_warnings
update => []
All other arguments are passed to update method.
update
Optionally load configuration data. You should design this config data to suppress any error or warning mentioned above. E.g:
load => 'binary:seaview Synopsis="multiplatform interface for sequence alignment"',
See Config::Model::Loader for the syntax of the string accepted by load parameter.
load
Optionally, run a check before running apply_fix (if any). This step is useful to check warning messages:
check_before_fix => { dump_errors => [ ... ] # optional, see below load4perl_dump_warnings => [ ... ] # optional, see below }
Use dump_errors if you expect issues:
dump_errors
check_before_fix => { dump_errors => [ # the issues and a way to fix the issue using Config::Model::Node::load qr/mandatory/ => 'Files:"*" Copyright:0="(c) foobar"', qr/mandatory/ => ' License:FOO text="foo bar" ! Files:"*" License short_name="FOO" ' ], }
Likewise, specify any expected warnings:
check_before_fix => { log4perl_dump_warnings => [ ... ], }
log4perl_dump_warnings passes the array ref content to expect function of Test::Log::Log4Perl.
log4perl_dump_warnings
Both log4perl_dump_warnings and dump_errors can be specified in check_before_fix hash.
check_before_fix
Optionally, call apply_fixes:
apply_fix => 1,
Call dump_tree to check the validity of the data after optional apply_fix. This step is not optional.
apply_fix
As with check_before_fix, both dump_errors or dump_warnings can be used.
dump_warnings
Run specific content check to verify that configuration data was retrieved correctly:
check => { 'fs:/proc fs_spec', "proc" , 'fs:/proc fs_file', "/proc" , 'fs:/home fs_file', "/home", },
The keys of the hash points to the value to be checked using the syntax described in "grab(...)" in Config::Model::AnyThing:.
You can run check using different check modes (See "fetch( ... )" in Config::Model::Value) by passing a hash ref instead of a scalar :
check => { 'sections:debian packages:0' , { mode => 'layered', value => 'dpkg-dev' }, 'sections:base packages:0', { mode => 'layered', value => "gcc-4.2-base' }, },
The whole hash content (except "value") is passed to grab and fetch
A regexp can also be used to check value:
check => { "License text" => qr/gnu/i, "License text" => { mode => 'custom', value => qr/gnu/i }, }
Verify if a hash contains one or more keys (or keys matching a regexp):
has_key => [ 'sections' => 'debian', # sections must point to a hash element 'control' => [qw/source binary/], 'copyright Files' => qr/.c$/, 'copyright Files' => [qr/\.h$/], qr/\.c$/], ],
Verify that a hash has not a key (or a key matching a regexp):
has_not_key => [ 'copyright Files' => qr/.virus$/ # silly, isn't ? ],
Verify annotation extracted from the configuration file comments:
verify_annotation => { 'source Build-Depends' => "do NOT add libgtk2-perl to build-deps (see bug #554704)", 'source Maintainer' => "what a fine\nteam this one is", },
Write back the config data in wr_root/model_tests/<subtest name>/. Note that write back is forced, so the tested configuration files are written back even if the configuration values were not changed during the test.
You can skip warning when writing back with the global :
no_warnings => 1,
Check the content of the written files(s) with Test::File::Contents. Tests can be grouped in an array ref:
file_contents => { "/home/foo/my_arm.conf" => "really big string" , "/home/bar/my_arm.conf" => [ "really big string" , "another"], , } file_contents_like => { "/home/foo/my_arm.conf" => [ qr/should be there/, qr/as well/ ] , } file_contents_unlike => { "/home/foo/my_arm.conf" => qr/should NOT be there/ , }
Check the mode of the written files:
file_mode => { "~/.ssh/ssh_config" => 0600, # octal mode "debian/stuff.postinst" => 0755, }
Only the last four octets of the mode are tested. I.e. the test is done with $file_mode & 07777
$file_mode & 07777
Note: this test is skipped on Windows
Check added or removed configuration files. If you expect changes, specify a subref to alter the file list:
file_check_sub => sub { my $list_ref = shift ; # file added during tests push @$list_ref, "/debian/source/format" ; },
Note that actual and expected file lists are sorted before check, adding a file can be done with push.
push
Copy all config data from wr_root/model_tests/<subtest name>/ to wr_root/model_tests/<subtest name>-w/. This steps is necessary to check that configuration written back has the same content as the original configuration.
wr_root/model_tests/<subtest name>-w/
Create a second configuration instance to read the conf file that was just copied (configuration data is checked.)
You can skip the load check if the written file still contain errors (e.g. some errors were ignored and cannot be fixed) with load_check2 => 'no'
load_check2 => 'no'
Optionally load configuration data in the second instance. You should design this config data to suppress any error or warning that occur in the step below. E.g:
load2 => 'binary:seaview',
See Config::Model::Loader for the syntax of the string accepted by load2 parameter.
load2
Compare data read from original data.
Run specific content check on the written config file to verify that configuration data was written and retrieved correctly:
wr_check => { 'fs:/proc fs_spec' => "proc" , 'fs:/proc fs_file' => "/proc", 'fs:/home fs_file' => "/home", },
Like the check item explained above, you can run wr_check using different check modes.
check
wr_check
Run all tests with one of these commands:
prove -l t/model_test.t :: [ t|l|e [ <model_name> [ <regexp> ]]] perl -Ilib t/model_test.t [ t|l|e [ <model_name> [ <regexp> ]]]
By default, all tests are run on all models.
You can pass arguments to t/model_test.t:
a bunch of letters. 't' to get test traces. 'e' to get stack trace in case of errors, 'l' to have logs. All other letters are ignored. E.g.
# run with log and error traces prove -lv t/model_test.t :: el
The model name to tests. E.g.:
# run only fstab tests prove -lv t/model_test.t :: x fstab
A regexp to filter subtest E.g.:
# run only fstab tests foobar subtest prove -lv t/model_test.t :: x fstab foobar # run only fstab tests foo subtest prove -lv t/model_test.t :: x fstab '^foo$'
LCDproc has a single configuration file: /etc/LCDd.conf. Here's LCDproc test layout and the test specification
Dpkg packages are constructed from several files. These files are handled like configuration files by Config::Model::Dpkg. The test layout features test with multiple file in dpkg-examples. The test is specified in dpkg-test-conf.pl
multistrap-test-conf.pl and multistrap-examples specify a test where the configuration file name is not imposed by the application. The file name must then be set in the test specification.
backend-shellvar-test-conf.pl is a more complex example showing how to test a backend. The test is done creating a dummy model within the test specification.
Config::Model
Test::More
Dominique Dumont
This software is Copyright (c) 2013-2018 by Dominique Dumont.
This is free software, licensed under:
The GNU Lesser General Public License, Version 2.1, February 1999
The following websites have more information about this module, and may be of help to you. As always, in addition to those websites please use your favorite search engine to discover more resources.
Search CPAN
The default CPAN search engine, useful to view POD in HTML format.
http://search.cpan.org/dist/Config-Model-Tester
AnnoCPAN
The AnnoCPAN is a website that allows community annotations of Perl module documentation.
http://annocpan.org/dist/Config-Model-Tester
CPAN Ratings
The CPAN Ratings is a website that allows community ratings and reviews of Perl modules.
http://cpanratings.perl.org/d/Config-Model-Tester
CPANTS
The CPANTS is a website that analyzes the Kwalitee ( code metrics ) of a distribution.
http://cpants.cpanauthors.org/dist/Config-Model-Tester
CPAN Testers
The CPAN Testers is a network of smoke testers who run automated tests on uploaded CPAN distributions.
http://www.cpantesters.org/distro/C/Config-Model-Tester
CPAN Testers Matrix
The CPAN Testers Matrix is a website that provides a visual overview of the test results for a distribution on various Perls/platforms.
http://matrix.cpantesters.org/?dist=Config-Model-Tester
CPAN Testers Dependencies
The CPAN Testers Dependencies is a website that shows a chart of the test results of all dependencies for a distribution.
http://deps.cpantesters.org/?module=Config::Model::Tester
Please report any bugs or feature requests by email to ddumont at cpan.org, or through the web interface at https://github.com/dod38fr/config-model-tester/issues. You will be automatically notified of any progress on the request by the system.
ddumont at cpan.org
The code is open to the world, and available for you to hack on. Please feel free to browse it and play with it, or whatever. If you want to contribute patches, please send me a diff or prod me to pull from your repository :)
http://github.com/dod38fr/config-model-tester.git
git clone git://github.com/dod38fr/config-model-tester.git
To install Config::Model::Tester, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Config::Model::Tester
CPAN shell
perl -MCPAN -e shell install Config::Model::Tester
For more information on module installation, please visit the detailed CPAN module installation guide.