Philip Crow > Bigtop-0.38 > Bigtop::Parser

Download:
Bigtop-0.38.tar.gz

Dependencies

Annotate this POD

CPAN RT

New  2
Open  1
View/Report Bugs
Source  

NAME ^

Bigtop::Parser - the Parse::RecDescent grammar driven parser for bigtop files

SYNOPSIS ^

Make a file like this:

    config {
        base_dir `/home/username`;
        Type1 Backend {}
        Type2 Backend {}
        Type3 Backend {}
    }
    app App::Name {
        table name { }
        controller SomeController {}
    }

Then run this command:

    bigtop my.bigtop all

DESCRIPTION ^

This module is really only designed to be used by the bigtop and tentmaker scripts. It provides access to the grammar which understands bigtop files and turns them into syntax trees. It provides various utility functions for bigtop, tentmaker, backends, and similar tools you might write.

If you just want to use bigtop, you should look in Bigtop::Docs::TOC where all the docs are outlined.

Reading further is an indication that you are interested in working on Bigtop and not just in using it to serve your needs.

METHODS ^

In this section, the methods are grouped, so that similar ones appear together.

METHODS which drive generation for scripts

gen_from_file

The bigtop script calls this method.

Returns: the app name and the name of the build directory.

You can call this as a class method passing it the name of the bigtop file to read and a list of the things to build.

The method is actually quite simple. It merely reads the file, then calls gen_from_string.

gen_from_string

The bigtop script calls this method when --new is used.

Returns: the app name and the name of the build directory.

This method orchestrates the build. It is called internally by gen_from_file. Call it as a class method. Pass it a hash with these keys:

    bigtop_string => the bigtop source code
    bigtop_file   => the name of the bigtop file, if you know it
    create        => are you in create mode? if so make this true
    build_list    => [ what to build ]
    flags         => command line args given to your script

bigtop_file is used by Bigtop::Init::Std to copy the bigtop file from its original location into the docs subdirectory of the build directory. If the file name is not defined, it skips that step.

If you set create to any true value, you will be in create mode and bigtop will make the build directory as a subdirectory of the current directory. Otherwise, it will make sure you are in a directory which looks like a build directory before building.

The list of things to build can include any backend type listed in the config block and/or the word 'all'. 'all' will be replaced with a list of all the backend types in the config section (in the order they appear there), as if they had been passed in.

It is legal to mention the same backend more than once. For instance, you could call gen_from_string directly

    Bigtop::Parser->gen_from_string(
        {
            bigtop_string => $bigtop_string,
            bigtop_file   => 'file.bigtop',
            create        => $create,
            build_list    => [ 'Init', 'Control', 'Init' ]
        }
    );

or equivalently, and more typically, you could call gen_from_file:

    Bigtop::Parser->gen_from_file(
        'file.bigtop', $create, 'Init', 'Control', 'Init'
    );

Either of these might be useful, if the first Init sets up directories that the Control backend needs, but the generated output from Control should influence the contents of file which Init finally builds. Check your backends for details.

The flags are given to Init Std as text, so they may be preserved for posterity in the Changes file.

gen_from_string internals

gen_from_string works like this. First, it attempts to parse the config section of the bigtop string. If that works, it iterates through each backend mentioned there building a list of modules to require. This includes looking in backend blocks for template statements. Their values must be template files relative to the directory from which bigtop was invoked.

Once the list is built, it calls its own import method to require them. This allows each backend to register its keywords. If any keyword used in the app section is not registered, a fatal parse error results.

Once the backends are all required, gen_from_string parses the whole bigtop string into an abstract syntax tree (AST). Then it iterates through the build list calling gen_Type on each element's backend. So this:

    config {
        Init Std      {}
        SQL  Postgres { template `postgres.tt`; }
    }
    app ...

    Bigtop::Parser->gen_from_string(
            $bigtop_string, 'file.bigtop', 'Init', 'SQL'
    );

Results first in the loading of Bigtop::Init::Std and Bigtop::SQL::Postgres, then in calling gen_Init on Init::Std and gen_SQL on SQL::Postgres. During the loading, setup_template is called with postgres.tt on SQL::Postgres.

gen_* methods are called as class methods. They receive the build directory, the AST, and the name of the bigtop_file (which could be undef). Backends can do whatever they like from there. Typically, they put files onto the disk. Those files might be web server conf files, sql to build the database, control modules, templates for viewing, models, etc.

METHODS which invoke the grammar

parse_config_string

Called as a class method (usually by gen_from_string), this method receives the bigtop input as a string. It attempts to parse only the config section which it returns as an AST. Syntax errors in the config section are fatal. Errors in the app section are not noticed.

parse_file

Call this as a class method, passing it the file name to read. It reads the file into memory, then calls parse_string, returning whatever it returns.

parse_string

Call this as a class method, passing it the bigtop string to parse. It calls the grammar to turn the input into an AST, which it returns.

METHODS which control which simple statement keywords are legal

add_valid_keywords

The grammar of a bigtop file is structured, but the legal keywords in its simple statements are defined by the backends (excepts that the config keywords are defined by this module, see Config Keywords below for those).

Acutally, all the keywords that any module will use should be defined in Bigtop::Keywords so tentmaker can display them. Then the backend (or its type) should pull the keyword definitions it wants from Bigtop::Keywords.

If you are writing a backend, you should use the base module for your backend type. This will register the standard keywords for that type. For example, suppose you are writing Bigtop::Backend::SQL::neWdB. It should be enough to say:

    use Bigtop::SQL;

in your module.

If you need to add additional keywords that are specific to your backend, put them in a begin block like this:

    BEGIN {
        Bigtop::Parser->add_valid_keywords(
            Bigtop::Keywords->get_docs_for(
                $type,
                qw( your keywords here),
            )
        );
    }

Here $type is the name of the surrounding block in which this keyword will make a valid statement. For example, if $type above is 'app' then this would be legal:

    app App::Name {
        your value;
    }

The type must be one of these levels:

config
app
app_literal
table
join_table
field
controller
controller_literal
method

These correspond to the block types in the grammar. Note, that there are also sequence blocks, but they are deprecated and never allowed statements. Further, the various literals are blocks in the grammar (they have block idents and can have defined keywords), but they don't have brace delimiters. Instead, they have a single backquoted string.

is_valid_keyword

Call this as a class method, passing it a type of keyword and a word that might be a valid keyword of that type.

Returns true if the keyword is valid, false otherwise.

get_valid_keywords

Call this as a class method passing it the type of keywords you want.

Returns a list of all registered keywords, of the requested type, in string sorted order.

The two preceding methogs are really for internal use in the grammar.

METHODS which work on the AST

There are quite a few other methods not documented here (shame on me). Most of those support tentmaker manipulations of the tree, but there are also some convenience accessors.

walk_postorder

Walks the AST for you, calling you back when it's time to build something.

The most common skeleton for gen_Backend is:

    use Bigtop;
    use Bigtop::Backend;

    sub gen_Backend {
        my $class     = shift;
        my $build_dir = shift;
        my $tree      = shift;

        # walk the tree
        my $something     = $tree->walk_postoder( 'output_something' );
        my $something_str = join '', @{ $something };

        # write the file
        Bigtop::write_file( $build_dir, $something_string );
    }

This walks the tree from the root. The walking is postorder meaning that all children are visited before the current node. Each walk_postorder returns an array reference (which is why we have to join the result in the above skeleton). After the children have been visited, the callback (output_something in the example) is called with their output array reference. You can also pass an additional scalar (which is usually a hash reference) to walk_postorder. It will be passed along to all the child walk_postorders and to the callbacks.

With this module walking the tree, all you must do is provide the appropriate callbacks. Put one at each level of the tree that interests you.

For example, if you are generating SQL, you need to put callbacks in at least the following packages:

    table_block
    table_element_block
    field_statement

This does require some knowledge of the tree. Please consult bigtop.grammar, in the lib/Bigtop subdirectory of Bigtop's build directory, for the possible packages (or grep for package on this file). There are also several chapters of the Gantry book devoted to explaining how to use the AST to build backends.

The callbacks are called as methods on the current tree node. They receive the output array reference from their children and the data scalar that was passed to walk_postorder (if one was passed in the top level call). So, a typical callback method might look like this:

    sub output_something {
        my $self         = shift;
        my $child_output = shift;
        my $data         = shift;
        ...
        return [ $output ];
    }

Remember that they must return an array reference. If you need something fancy, you might do this:

    return [ [ type1_output => $toutput, type2_output => $other_out ] ];

Then the parent package's callback will receive that and must tease apart the the two types. Note that I have nested arrays here. This prevents two children from overwriting each other's output if you are ever tempted to try saving the return list directly to a hash (think recursion).

(walk_postorder also passes the current node to each child after the data scalar. This is the child's parent, which is really only useful during parent building inside the grammar. The parent comes after the data scalar in both walk_postorder and in the callback. Most backends will just peek in $self->{__PARENT__} which is gauranteed to have the parent once the grammar finishes with the AST.)

set_parent

This method is the callback used by the grammar to make sure that all nodes know who their daddy is. You shouldn't call it, but looking at it shows what the simplest callback might look like. Note that there is only one of these and it lives in the application_ancestor package, which is not one of the packages defined in the grammar. But, this module makes sure that all the grammar defined packages inherit from it.

build_lookup_hash

This method builds the lookup hash you can use to find data about other parts of the tree, without walking to it.

The AST actually has three keys: configuration, application, and lookup. The first two are built in the normal way from the input file. They are genuine ASTs in their own right. The lookup key is not. It does not preserve order. But it does make it easier to retrieve things.

For example, suppose that you are in the method_body package attempting to verify that requested fields for this method are defined in the table for this controller. You could walk the tree, but the lookup hash makes it easier:

    unless (
        defined $tree->{lookup}{tables}{$table_name}{fields}{$field_name}
    ) {
        die "No such column $field_name\n";
    }

The easiest way to know what is available is to dump the lookup hash. But the pattern is basically this. At the top level there are fixed keywords for the app level block types: tables, sequences, controllers. The next level is the name of a block. Under that, there is a fixed keyword for each subblock type, etc.

METHODS for use in walk_postorder callbacks

dumpme

Use this method instead of directly calling Data::Dumper::Dump.

While you could dump $self, that's rather messy. The problem is the parent nodes. Their presence means a simple dump will always show the whole app AST. This method carefully removes the parent, dumps the node, and restores the parent, reducing clutter and leaving everything in tact. The closer to a leaf you get, the better it works.

get_appname

Call this on the full AST. It returns the name of the application.

get_config

Call this on the full AST. It returns the config subtree.

get_controller_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the controller for this method. This is useful for error reporting.

get_method_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of this method. Useful for error reporting.

get_name

While this should work everywhere, it doesn't. Some packages have it. If yours does, call it. Otherwise peek in $self->{__NAME__}. But, remember that not everything has a name.

get_table_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the table this controller controls. Useful for error reporting.

METHODS used internally

import

You probably don't need to call this. But, if you do, pass it a list of backends to import like this:

    use Bigtop::Parser qw( Type=Backend=template.tt );

This will load Bigtop::Type::Backend and tell it to use template.tt. You can accomplish the same thing by directly calling import as a class method:

    Bigtop::Parser->import( 'Type=Backend=template.tt' );
fatal_error_two_lines

This method is used by the grammar to report fatal parse error in the input. It actually gives 50 characters of trailing context, not two lines, but the name stuck.

fatal_keyword_error

This method is used by the grammer to report on unregistered (often misspelled) keywords. It identifies the offending keyword and the line where it appeared in the input, gives the remainder of the line on which it was seen (which is sometimes only whitespace), and lists the legal choices (often wrapping them in an ugly fashion).

Config KEYWORDS ^

For simplicity, all config keywords are requested from Bigtop::Keywords in this module. This is not necessarily ideal and is subject to change.

base_dir

Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the build directory will be made as a subdirectory of the base_dir. For instance, I could use my home directory:

    base_dir `/home/username`;

Note that you need the backquotes to hide the slashes. Also note, that you should use a path which looks good on your development system. In particular, this would work on the appropriate platform:

    base_dir `C:\path\to\build`;

The default base_dir is the current directory from which bigtop is run.

app_dir

Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the actual generated files will be placed into base_dir/app_dir (where the slash is correctly replaced with your OS path separator). If you are in create mode, but don't supply an app_dir, a default is formed from the app name in the manner h2xs would use. Consider:

    config {
        base_dir `/home/username`;
    }
    app App::Name {
    }

In this case the app_dir is App-Name. So the build directory is

    /home/username/App-Name

By specifying your own app_dir statement, you have complete control of where the app is initially built. For example:

    config {
        base_dir `/home/username`;
        app_dir  `myappdir`;
    }
    app App::Name { }

Will build in /home/username/myappdir.

When not using create mode, all files will be built under the current directory. If that directory doesn't look like an app build directory, a fatal error will result. Either move to the proper directory, or use create mode to avoid the error.

engine

This is passed directly to the use Framework; statement of the top level controller.

Thus,

    engine MP13;

becomes something like this:

    use Framework qw/ engine=MP13 /;

in the base level controller. Both Catalyst and Gantry expect this syntax.

The available engines depend on what the framework supports. The one in the example is mod_perl 1.3 in the syntax of Catalyst and Gantry.

template_engine

Similar to engine, this specifies the template engine. Choices almost always include TT, but might also include Mason or other templaters depending on what your framework supports..

Other KEYWORDS ^

literal

This keyword applies to many backends at the app level and at some other levels. This keyword is special, because it expects a type keyword immediately before its values. For example:

    literal SQL `CREATE...`;

It always instructs someone (the backend of type SQL in the example) to directly insert the backquoted string into its output, without so much as adjusting whitespace.

Backend types that should obey this statement are:

    SQL      - for backends of type SQL
    Location - for backends constructing apache confs or the like

The literal Location statement may also be used at the controller level.

no_gen

Applies to backend blocks in the config block, app blocks, controller blocks, and method blocks.

gen_from_string enforces the app level no_gen. If it has a true value only a warning is printed, nothing is generated. None of the backends are called.

gen_from_string also enforces no_gen on entire backends, if their config block has a true no_gen value.

The Control backend of your choice is responsible for enforcing no_gen at the controller and method levels.

not_for

Applies to tables and fields (although the latter only worked for Models at the time of this writing).

Each backend is responsible for enforcing not_for. It should mean that the field or table is ignored by the named backend type. Thus

    table skip_model {
        not_for Model;
    }

should generate as normal in SQL backends, but should be completely ignored for Models. The same should hold for fields marked not_for. But my SQL backends didn't do that when I wrote this, only the Models worked.

METHODS ^

get_keyword_docs

Called by TentMaker, so it can display the backend comments to the user through their browser.

Returns: a hash reference of keyword docs understood by tentmaker's templates.

gen_mode

Used internally.

Get accessor for whether we are really generating, or just serving tentmaker. If we are not generating, there is no need to set up the templates for all the backends.

set_gen_mode

Used internally.

Set accessor for whether we are really generating.

get_ident

Returns: the next available ident (as ident_n).

get_parser

Used internally.

Accessor to ensure that only one parser is ever instantiated.

load_backends

Used internally.

Responsible for loading all needed backends.

preprocess

Used internally.

Strips comment lines.

Returns: a hash keyed by line number, storing the comment on that line before it was stripped..

AUTHOR ^

Phil Crow <crow.phil@gmail.com>

COPYRIGHT and LICENSE ^

Copyright (C) 2005-7 by Phil Crow

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.6 or, at your option, any later version of Perl 5 you may have available.

syntax highlighting: