benchmark_template_engines - Test the relative performance of several different types of template engines.
Options: --help brief help message --man full documentation --duration=N,-d N duration (in seconds) to run each benchmark (default 30) --repeats=N,-r N number of repeated sections in template (default 30) --dataset=S which presupplied dataset to use (default 'original') --progress, --noprogress show or hide a progress bar (default hide) --json, --nojson enable or disable JSON-formatted output (default off) --quiet, --noquiet show or hide extra output beyond the report (default show) --keeptmpdirs, --nokeeptmpdirs keep or remove template/cache dirs (default remove) --skipoutputcompare, --noskipoutputcompare skip the check that template outputs matched (default don't skip) --style=S style to pass to Benchmark.pm (default 'none') --<feature>, --no<feature> enable or disable a Template::Benchmark feature --<feature>=N enable a feature with a specific number of repeats --<cache_type>, --no<cache_type> enable or disable a Template::Benchmark cache type --featuresfrom=plugin enable all features supported by that plugin --typesfrom=plugin enable all cache types supported by that plugin --all enable all features and cache types --allfeatures enable all teamplate features --nofeatures disable all template features --alltypes enable all cache types --notypes disable all cache types --featurematrix display matrix of what engines support what features -I <directory> push <directory> onto perl's module search path
benchmark_template_engines --progress --notypes --uncached_string \ --onlyplugin TemplateSandbox,HTMLTemplate,TemplateToolkit \ --onlyplugin TemplateAlloyTT,TemplateAlloyHT --- Starting Benchmarks -------------------------------------------------------- ETA: 7 benchmarks to run = 70 seconds minimum. progress: 100% [=====================================================]D 0h01m25s --- Template Benchmark @Thu Feb 18 20:14:08 2010 ------------------------------- HT - HTML::Template (2.9) TAHT - Template::Alloy (1.013) in HTML::Template mode TATT - Template::Alloy (1.013) in Template::Toolkit mode TS - Template::Sandbox (1.02_01) without caching TT - Template::Toolkit (2.22) TT_X - Template::Toolkit (2.22) with Stash::XS (no version number) TT_XCET - Template::Toolkit (2.22) with Stash::XS (no version number) and Template::Parser::CET (0.05) --- uncached_string ------------------------------------------------------------ Rate TT TT_X TATT TT_XCET TS TAHT HT TT 4.69/s -- -16% -56% -56% -66% -73% -80% TT_X 5.60/s 19% -- -47% -48% -59% -68% -76% TATT 10.6/s 125% 88% -- -2% -23% -39% -54% TT_XCET 10.7/s 129% 92% 2% -- -21% -38% -54% TS 13.6/s 191% 144% 29% 27% -- -21% -41% TAHT 17.2/s 268% 208% 63% 61% 26% -- -26% HT 23.1/s 394% 314% 119% 116% 70% 34% --
benchmark_template_engines builds a test template according to various commandline parameters (or some "sensible" defaults) and then benchmarks the performance of a number of templating systems at running the template.
It groups the template systems into three main categories for the purposes of comparing like-with-like: those tests that are from a string template held in memory and parsed/compiled on each execution; those that read the template or a compiled version from file on every execution; and those that read the template from file first and then hold the template or a compiled version in memory.
These are roughly analogous to running in a plain CGI environment without caching; running in CGI using file caching; and running under mod_perl with memory caching.
These results may bear no resemblence to real-world performance in absolute terms, however they should provide an indication of the strengths and weaknesses of the different template systems relative to each other.
For more details on how the benchmarks are performed and on various options, consult the documentation for Template::Benchmark.
Run the benchmark of each template engine for N seconds. Increasing this will reduce the statistical error on the benchmark results. It will also take the benchmark stript longer to run of course.
If the duration is set to 0 on the commandline then no benchmarks will be timed, however the initial run to test template output will still happen, which may be useful to developers writing new plugins. (Default: 10 seconds.)
Repeat the central section of the template N times to simulate a longer template. Tweak this if you want to see behaviour over particularly long or short templates. (Default: 30 repeats.)
Provide the name of one of the presupplied datasets to use instead of the default. (Default: 'original'.)
If enabled, this will display a progress bar using Term::ProgressBar::Simple during the benchmark run. The progress bar is only updated between benchmark runs, so while it won't effect timings it will only update every 10 seconds (or whatever you've passed to --duration).
If benchmark_template_engines is run non-interactively or if Term::ProgressBar::Simple is not installed, no progress bar will be displayed. (Default: disabled.)
If enabled this option will suppress all output except that of the benchmark report. This includes suppressing any reasons for template engines being skipped. (Default: disabled, ie: show extra stuff.)
If enabled this option will output the report as a data-structure encoded using JSON::Any. This could be suitable for storage so that you can record historical benchmark data and read it with a program later.
If you set --json then --quiet will automatically be set also, to prevent extra output messing up the JSON output. (Default: disabled.)
If enabled, this option will skip the removal of the temporary dirs created to store the generated templates and caches, and on program exit it will print the location of these temporary dirs.
This is useful if you wish to inspect the dirs for debugging purposes.
Note that the output of the dir locations does NOT respect the --quiet option, so if --keeptmpdirs is used in conjunction with --json the output of the directory locations will most likely corrupt the format of your JSON output. (Default: disabled, ie: remove dirs.)
If enabled, this option will skip the sanity check that compares the output of the different template engines to ensure that they're producing the same output.
This is useful as a workaround if the sanity check is producing a mismatch error that you deem to be unimportant. It is strongly recommended that you never use this option without first manually checking the mismatched outputs to be certain that they are in fact unimportant.
(Default: disabled, ie: perform the check.)
This option may be useful if you want to see the raw timings as Benchmark produces them.
If you set the style to anything other than 'none', it will ignore the --quiet option, and will corrupt any --json enabled output. (Default: none.)
Enable or disable the named feature within Template::Benchmark when producing the benchmark template.
Enable or disable the type of caching within Template::Benchmark when producing the benchmarks.
Valid values are: uncached_string, uncached_disk, file_cache, shared_memory_cache, memory_cache and instance_reuse. See the documentation for Template::Benchmark for an accurate and up-to-date listing of values and what they mean. (Default: whatever Template::Benchmark sets as defaults.)
Enable all available cache types and template features except those you have explicitly disabled with their corresponding --no<feature> switch.
Disable all available template features or cache types except those you have explicitly enabled with their corresponding --<feature> or --<type> switch.
Enable all available template features or cache types except those you have explicitly disabled with their corresponding --no<feature> or --no<type> switch.
Sets the features or cache types to be enabled to be all those supported by the given plugin.
Setting either of these options will override all other options for enabling or disabling either features or cache types.
Print a chart showing a matrix of all template engines vs the chosen template features with a Y or N indicating support for that feature by that engine.
If you wish to see a matrix of all features, make certain to use the --all or --allfeatures options too.
Once the feature matrix is displayed the program exits and no benchmarks will be run.
Print a chart showing a matrix of all template engines against some common types of template engine syntax with a Y or N indicating if it could be described in that fashion.
The current syntax types are: embedded-perl, mini-language. See the Template::Benchmark::Engine documentation for more information on what these types represent.
A column indicating whether a template engine is implemented in pure-perl is also displayed.
Once the syntax matrix is displayed the program exits and no benchmarks will be run.
This will push directory onto perl's module path with
use lib '<directory>', this is handy if you want to use a development copy of a plugin or template engine that isn't installed to the usual perl paths.
This will push plugin name onto the whitelist or blacklist of template engine plugins to run.
Print a brief help message and exits.
Prints the manual page and exits.
Thanks to Paul Seamons for creating the the bench_various_templaters.pl script distributed with Template::Alloy, which was the ultimate inspiration for this script.
<libtemplate-benchmark-perl at illusori.co.uk>
Copyright 2010 Sam Graham.
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.