The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Pistachio-0.01
River stage zero No dependents
3 ++
/ Pistachio::Tokenizer

NAME

Pistachio::Tokenizer - provides iterator(), which turns source code text into a Pistachio::Token iterator

VERSION

version 0.01

SYNOPSIS

 use Pistachio::Tokenizer;
 my $tokenizer = Pistachio::Tokenizer->new('Perl5');

 my $scalar_ref = \"use strict; ...;";
 my $it = $tokenizer->iterator($scalar_ref);
 
 while (my $token = $it->()) {
     print $token->type, ': ', $token->value, "\n";
 }

AUTHOR

Joel Dalley <joeldalley@gmail.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2013 by Joel Dalley.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.