===============================================================================
|     ______                           ______              ______             |
|   /\   __ \                         /\__  _\            /\__  _\            |
|   \ \  \/\ \ __  __  ______  ______ \/_/\ \/ ______  ___\/_/\ \/ ______     |
|    \ \   __//\ \/\ \/\  __ \/\  ___\   \ \ \/\  ___\/\  ___\ \ \/\  ___\    |
|     \ \  \/ \ \ \_\ \ \ \/\ \ \___  \   \ \ \ \  ___\ \___  \ \ \ \___  \   |
|      \ \__\  \ \____/\ \____ \/\_____\   \ \_\ \_____\/\_____\ \_\/\_____\  |
|       \/__/   \/___/  \/___/\ \/____/     \/_/\/____/ \/____/ \/_/\/____/   |
|                         /\____/                                             |
|                         \/___/                                              |
|                                                                             |
===============================================================================
 
                     Welcome to the Pugs Test suite. 
 
Pugs is currently being developed in a highly test-driven manner. Tests are 
written for both implemented and unimplemented features and are based roughly
on the Perl6 Language Synopsis documents. All are welcome and encouraged to
contribute to this test suite. It is the hope that not only will this test
suite be used for Pugs, but that it will eventually become the Perl6 test
suite itself.

-------------------------------------------------------------------------------
Getting Started
-------------------------------------------------------------------------------

Here are some basic guidelines to help you get started writing tests for Pugs. 

- If you are unsure of something, don't hesitate to ask.

If you have a question about a test, written or unwritten, log on to #perl6
or send an email to perl6-compiler and ask someone about it. If you are 
unsure about a perl6 language element, we encourage you to email the 
perl6-language list and get clarification. Pugs and Perl6 are group efforts
and asking questions is a good thing.

- What to test

A number of Pugs hackers on #perl6 run regular smoke tests, and you can run 
your own using 'make smoke'. The smoke test produces an HTML graph of what tests
are passing, and what aren't. This can be a good place to start.

There is also a cross linking which is made between the tests and the synopses.
It's generated by running the 'util/catalog_tests.pl' script.

- Use the Test module.

We have a created a basic Test module found in ext/Test/lib/Test.pm. It is 
written in Perl 6 and implements the TAP protocol (and so can be used with 
Test::Harness). The module has its own documentation and I encourage you to 
read it. 

- Pugs tests should have a she-bang line of #!/usr/bin/pugs.

This line helps both Test::Harness as well as the 'prove' utility when 
running Pugs tests. 

- Place tests in the appropriate folder.

We have recently undergone a re-organization of the test suite in order to 
make it easier to find what has and has not been tested. It is important as 
the test suite grows that we try to keep this organization. If you have a test
and are unsure of where to put it, ask on #perl6 for help, or put it in the
general/ folder and email perl6-compiler and let us know.

- If possible, please backlink your tests to the Synopsis

Backlinks are planted in the test file, and point to the appropriate sections
of the Synopsis you are using to write the test.

The look like pod links:

	L<S06/Blocks> # "S06" is synopsis 6, and "Blocks" is the section
	L<S03/"Hyper operators"> # quotes are used when spaces are in the title

The section name should be copied verbatim from the pod.

The links also have a weird extension: you can specify a regex, to skip forward
from the linked section, so the backlink is put in a more specific place:

	L<<S05/"Return values from matches" /In numeric context.*number of matches:/>>

The above link is appropriate next to a test case checking rule application in
numeric context, and it will place the backlink appropriately.

Try running 'grep -r "L<" t/' to see some examples.

- Dealing with parse failures

When developing tests for features that have not been implemented yet,
often you find yourself writing code that doesn't compile. Don't get
stuck on this: wrap the new code with eval and test it anyway. Just make
sure that the test fails as long as the eval does, and until the feature
has been implemented correctly. You can use the eval_* functions for this.

Sometimes code is so futuristic, it can even confuse eval. We call this a
"hard parsefail". When this happens, comment out the failing code, but mark
it so it doesn't get forgotten, like so:

    fail("FIXME parsefail", :todo);
	#eval_ok('my code here');
	
Or another alternate style is this:

    eval_ok('# $code.which(%will, @fail)');
    
Which essentially comments out your eval, and returns 'undef' to eval_ok().

- When TODO and when not TODO.

All of the functions in the Test module also have a named 'todo' parameter. 
The general rule about todo tests is that if the feature is not
yet implemented, it is TODO. But if a feature is broken, or a bug is found
then the tests should fail and *not* be TODO.

The only exception to this rule is that we TODO all failing tests before 
each weekly release. This is so 'make test' will succeed :)

Remember, the failing test *is* your bug report.