NAME
Dist::Zilla::Plugin::Test::Compile::PerFile - Create a single .t for
each compilable file in a distribution
VERSION
version 0.001001
SYNOPSIS
; in dist.ini
[Test::Compile::PerFile]
DESCRIPTION
This module is inspired by its earlier sibling "[Test::Compile]".
Test::Compile is awesome, however, in the process of its development, we
discovered it might be useful to run compilation tests in parallel.
This lead to the realization that implementing said functions are kinda
messy.
However, a further realization is, that parallelism should not be
codified in the test itself, because platform parallelism is rather not
very portable, so parallelism should only be enabled when asked for.
And this lead to the realization that "prove" and "Test::Harness"
ALREADY implement parallelism, and ALREADY provide a safe way for
platforms to indicate parallelism is wanted.
Which means implementing another layer of parallelism is unwanted and
unproductive effort ( which may be also filled with messy
parallelism-induced bugs )
So, here is the Test::Compile model based on how development is
currently proceeding.
prove
\ ----- 00_compile.t
| \ ----- Compile Module 1
| \ ----- Compile Module 2
|
\ ----- 01_basic.t
That may be fine for some people, but this approach has several
fundamental limits:
1. Sub-Tasks of compile don't get load balanced by the master harness.
2. Parallelism is developer side, not deployment side governed.
3. This approach means "prove -s" will have no impact.
4. This approach means "prove -j" will have no impact.
5. This approach inhibits other features of "prove" such as the
"--state=slow"
So this variation aims to employ one test file per module, to leverage
"prove" power.
One initial concern cropped up on the notion of having excessive numbers
of "perl" instances, e.g:
prove
\ ----- 00_compile/01_Module_1.t
| \ ----- Compile Module 1
|
\ ----- 00_compile/02_Module_2.t
| \ ----- Compile Module 2
|
\ ----- 01_basic.t
If we were to implement it this way, we'd have the fun overhead of
having to spawn 2 "perl" instances per module tested, which on "Win32",
would roughly double the test time and give nothing in return.
However, Most of the reason for having a "perl" process per compile, was
to separate the modules from each other to assure they could be loaded
independently.
So because we already have a basically empty compile-state per test, we
can reduce the number of "perl" processes to as many modules as we have.
prove
\ ----- 00_compile/01_Module_1.t
|
\ ----- 00_compile/02_Module_2.t
|
\ ----- 01_basic.t
Granted, there is still some bleed here, because doing it like this
means you have some modules preloaded prior to compiling the module in
question, namely, that "Test::*" will be in scope.
However, "testing these modules compile without "Test::" loaded" is not
the real purpose of the compile tests, the compile tests are to make
sure the modules load.
So this is an acceptable caveat for this module, and if you wish to be
distinct from "Test::*", then you're encouraged to use the much more
proven "[Test::Compile]".
Though we may eventually provide an option to spawn additional "perl"
processes to more closely mimic "Test::*"'s behaviour, the cost of doing
so should not be understated, and as this module exist to attempt to
improve efficiency of tests, not to decrease them, that would be an
approach counter-productive to this modules purpose.
METHODS
"gather_files"
This plugin operates ONLY during "gather_files", unlike other plugins
which have multiple phase involvement, this only happens at this phase.
The intrinsic dependence of this plugin on other files in your dist,
means that in order for it to generate a test for any given file, the
test itself must be included after that file is gathered.
ATTRIBUTES
"xt_mode"
*optional* "Bool"
xt_mode = 1
If set, "prefix" defaults to "xt/author/00-compile"
*Default* is NOT SET
"prefix"
*optional* "Str"
prefix = t/99-compilerthingys
If set, sets the prefix path for generated tests to go in.
*Defaults* to "t/00-compile"
"file"
*optional* "multivalue_arg" "ArrayRef[Str]"
"mvp_aliases": "files"
file = lib/Foo.pm
file = lib/Bar.pm
files = lib/Quux.pm
file = script/whatever.pl
Specifies the list of source files to generate compile tests for.
*If not specified*, defaults are populated from the file finder "finder"
"skip"
*optional* "multivalue_arg" "ArrayRef[Str]"
skip = lib/Foo.pm
Specifies the list of source files to skip compile tests for.
"finder"
*optional* "multivalue_arg" "ArrayRef[Str]"
finder = :InstallModules
Specifies a "FileFinder" plugin name to query for a list of files to
build compile tests for.
*If not specified*, a custom one is autovivified, and matches only
"*.pm" in "lib/"
"path_translator"
*optional* "Str"
A Name of a routine to translate source paths ( i.e: Paths to
modules/scripts that are to be compiled ) into test file names.
*Default* is "base64_filter"
Supported Values:
* "base64_filter"
Paths are mangled so that they contain only base64 web-safe elements
<http://tools.ietf.org/html/rfc3548#section-4>
That is to say, if you were building tests for a distribution with
this layout:
lib/Foo/Bar.pm
lib/Foo.pm
lib/Foo_Quux.pm
That the generated test files will be in the "prefix" directory
named:
lib_Foo_Bar_pm.t
lib_Foo_pm.t
lib_Foo_Quux.t
This is the default, but not necessarily the most sane if you have
unusual file naming.
lib/Foo/Bar.pm
lib/Foo_Bar.pm
This configuration will not work with this translator.
* "mimic_source"
This is mostly a 1:1 mapping, it doesn't translate source names in
any way, other than prefixing and suffixing, which is standard
regardless of translation chosen.
lib/Foo/Bar.pm
lib/Foo.pm
lib/Foo_Quux.pm
Will emit a prefix directory populated as such
lib/Foo/Bar.pm.t
lib/Foo.pm.t
lib/Foo_Quux.pm.t
Indeed, if you had a death wish, you could set "prefix = lib" and
your final layout would be:
lib/Foo/Bar.pm
lib/Foo/Bar.pm.t
lib/Foo.pm
lib/Foo.pm.t
lib/Foo_Quux.pm
lib/Foo_Quux.pm.t
Though this is not advised, and is only given for an example.
"test_template"
Contains the string of the template file you wish to use as a reference
point.
Unlike most plugins, which use "Data::Section" to provide their
templates, this plugin uses a "File::ShareDir" "dist_dir" to distribute
templates.
This means there will always be a predetermined list of templates
shipped by this plugin, however, if you wish to modify these templates
and store them with a non-colliding name, for your personal convenience,
you are entirely free to so.
As such, this field takes as its parameter, the name of any file that
happened to be in the "dist_dir" at compile time.
Provided Templates:
* "01-basic.t.tpl"
A very basic standard template, which "use"'s "Test::More", does a
"requires_ok($file)" for the requested file, and nothing else.
Other Important Differences to Test::Compile
Finders useful, but not required
"[Test::Compile::PerFile]" supports providing an arbitrary list of files
to generate compile tests
[Test::Compile::PerFile]
file = lib/Foo.pm
file = lib/Quux.pm
Using this will supersede using finders to find things.
Single finder only, not multiple
"[Test::Compile]" supports 2 finder keys, "module_finder" and
"script_finder".
This module only supports one key, "finder", and it is expected that if
you want to test 2 different sets of files, you'll create a separate
instance for that:
-[Test::Compile]
-module_finder = Foo
-script_finder = bar
+[Test::Compile::PerFile / module compile tests]
+finder = Foo
+[Test::Compile::PerFile / script compile tests]
+finder = bar
This is harder to do with "[Test::Compile]", because you'd have to
declare a separate file name for it to work, where-as
"[Test::Compile::PerFile]" generates a unique file name for each source
it tests.
Collisions are still possible, but harder to hit by accident.
File Oriented, not Module Oriented
Under the hood, "Test::Compile" is really file oriented too, it just
doesn't give that impression on the box.
It just seemed fundamentally less complex to deal only in file paths for
this module, as it gives no illusions as to what it can, and cannot do.
( For example, by being clearly file oriented, there's no ambiguity of
how it will behave when a file name and a module name are miss-matching
in some way, by simply not caring about the latter , it will also never
attempt to probe and load modules that can't be automatically resolved
to files )
Performance
A rough comparison on the "dzil" git tree, with "HARNESS_OPTIONS=j4:c"
where 4 is the number of logical "CPUs" I have:
Test::Compile - Files= 42, Tests=577, 57 wallclock secs ( 0.32 usr 0.11 sys + 109.29 cusr 11.13 csys = 120.85 CPU)
Test::Compile::PerFile - Files=176, Tests=576, 44 wallclock secs ( 0.83 usr 0.39 sys + 127.34 cusr 13.27 csys = 141.83 CPU)
So a 20% saving for a 300% growth in file count, a 500k growth in
unpacked tar size, and a 4k growth in "tar.gz" size.
Hmm, that's a pretty serious trade off. Might not really be worth the
savings.
Though, comparing compile tests alone:
# Test::Compile
prove -j4lr --timer t/00-compile.t
Files=1, Tests=135, 41 wallclock secs ( 0.07 usr 0.01 sys + 36.82 cusr 3.58 csys = 40.48 CPU)
# Test::Compile::PerFile
prove -j4lr --timer t/00-compile/
Files=135, Tests=135, 22 wallclock secs ( 0.58 usr 0.32 sys + 64.45 cusr 6.74 csys = 72.09 CPU)
That's not bad, considering that although I have 4 logical "CPUs",
that's really just 2 physical "CPUs" with hyper-threading ;)
AUTHOR
Kent Fredric <kentfredric@gmail.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2013 by Kent Fredric
<kentfredric@gmail.com>.
This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.