גאבור סבו - Gábor Szabó > Test-Snapshots > Test::Snapshots

Download:
Test-Snapshots-0.02.tar.gz

Dependencies

Annotate this POD

View/Report Bugs
Module Version: 0.02   Source  

NAME ^

Test::Snapshots - for testing stand alone scripts and executables

SYNOPSIS ^

 use Test::More;
 use Test::Snapshots;

 test_all_snapshots('eg');

Will go over all the .pl files in the eg/ directory, run them using with the content of the SCRIPT.out and SCRIPT.err files

Optional configurations before calling test_all_snapshots:

 Test::Snapshots::debug(1);

Get some extra diag messages

 Test::Snapshots::combine(1);

Combines the stdout and stderr and compares them to the SCRIPT.out file

 Test::Snapshots::set_glob('*.t');

Change the way we locate the scripts to be executed.

 Test::Snapshots::set_accessories_dir('path/to/dir');

Change the place where TS looks for .out files.

WARNING ^

This is alpha software. The API will most certainly change as the requirements get clearer.

Examples ^

Many of the unit test of this module are actually simple use cases with the files to be tested located in the eg/ subdirectory of the distribution. Check them out.

TODO ^

DESCRIPTION ^

Test::Snapshots was created especially to be able to test a large number of command line oriented executables. It does not matter if the executable is something compiled from C, a Perl, Python or PHP script.

Test::Snapshot can be seen as a very simple replacement of Expect. It will go over the designated direcory and run every execute like this:

  executable arguments < input_file > output_file 2> error_file

It will then check if the output_file is the same as the exepcted output file and if the error_file is the sameas the expected error file.

If an input file is not supplied then the < input_file part will be omitted.

The input file, the list of arguments and the expected output and error files all have the same name as the executable. So if you have an executable called fabricate.exe then you'd create the following files:

  fabricate.exe.in
  fabricate.exe.argv
  
  fabricate.exe.out
  fabricate.exe.err
  fabricate.exe.exit

If .in is omitted we assume there is no input

If .argv is omitted then no arguments are provided

If .err or .out is omitted then it is assumed to be empty.

If .exit is omitted then it is expected that the exit code will be equal to the default exit code which is 0.

Multiple test cases

Sometime a single executable file should have multiple test cases. That is we might want to provide different .in and .argv files and expect different .out/.err/.exit values.

In order to allow such mode the files need to have a number in their name. So if you are testing xyz the files need to be

 xyz.01.in
 xyz.01.out
 
 xyz.02.in
 xyz.02.out
 xyz.02.err

The expected number of test is the number of different numbers so if you have two files xyz.01.in and xyz.27.err then Test::Snapshots will run two test. One of them has no input and some expected error while the other has only input and not expected output or error.

Timeout

In order to avoid stuck test cases (e.g. waiting on STDIN) by default every test case can run up to 10 secs.

METHODS ^

timeout

Set timeout for the executions so if one of them gets stuck (e.g. waiting on STDIN) the whole test suit won't suffer.

Default 10 secs.

combine

Set to 1 if you'd like to combine the STDOUT and STDERR and compare the combined output to the .out file.

Default is 0 meaning they will be captured separatelly and compared separatelly to the .out and .err files.

set_glob

Set what glob to use to fine the files to be executed. Currently it defaults to '*.pl' but maybe it should have no default forcing the user to set one.

skip

Pass to it a hash ref of path => 'explanation' pairs for all the files that need to be skipped.

  skip({
    path => 'good reason',
    path2 => 'some excuse',
  });

set_accessories_dir

We are calling the .out, .err etc files accessories.

In some cases you don't want them to be next to the script that are being tested. In such cases you can use the above function to tell Test::Snapshots where those files can be found.

command

By default Test::Snapshots will assume the files to be tested are stand alone executables or that at least they know where their interpreter is. So they will be executed directly.

In most of the cases you will want to run them with some specific command. e.g. You might want to make sure they run with the same perl interpreter as your test script runs. In that case call the following:

 command($^X)

In other cases the files need to be executed with some other tool, eg. the perl 6 or python interpreter which is in the path:

 command("perl6");

or

 command("python");

default_expected_exit_code

The exepceted exit code can be defined on a perl case basis in the .exit file. If the .exit file does not exist then there is a default expected exit code. Which is 0 by default.

Use this method to chane the default.

debug

You can turn on the debug flag by calling debug(1). If it is set Test::Snippets will call diag() with all kinds of information during the test execution.

test_all_snapshots

This is the call that actually goes out, locates all the files to be tested, sets the plan and executes all the test. Currently one should give a directory as a paramter to it but I plan to move that parameter to a separate method and to allow the setting of multiple directories.

test_single_file

Testing a single file. It gets the path to the file to be tested. The length of the prefix and optionally a case which is the 01, 02 etc. name of the test case for the multple-test-cases.

Currently this is considered an internal method.

See Also ^

Test::Simple, Test::More and Test::Most.

Test::Output, Capture::Tiny, Test::Cmd,

COPYRIGHT ^

Copyright 2009 Gabor Szabo gabor@szabgab.com http://szabgab.com/

LICENSE ^

This program is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself.

DISCLAIMER OF WARRANTY ^

BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR, OR CORRECTION.

IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

syntax highlighting: