The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
=pod

=head1 NAME

RPC::ExtDirect::Intro - A gentle(ish) introduction to RPC::ExtDirect

=head1 DESCRIPTION

=head2 What is Ext.Direct?

Ext.Direct is a high level RPC-over-HTTP protocol provided out of the box
with L<Ext JS|http://www.sencha.com/products/extjs/> and
L<Sencha Touch|http://www.sencha.com/products/touch/> JavaScript frameworks.
It is server agnostic, provided that the server side stack is conformant
to the
L<Ext.Direct specification|http://www.sencha.com/products/extjs/extdirect/>.

L<RPC::ExtDirect> is a fully featured implementation of Ext.Direct server
stack in Perl, compatible with Ext JS 4.x and 5.x, and Sencha Touch 2.x.

=head2 What Ext.Direct is for?

The primary goal for Ext.Direct is easy and fast integration of server
components with HTML5 applications. Client side stack is built in frameworks'
cores and is used by many components like data Stores, Forms, Grids, Charts,
etc. Ext.Direct supports request batching, file uploads, event polling and
many other features.

Besides simplicity and ease of use, Ext.Direct allows to achieve very clean
code and issue separation both on server and client sides, which in turn
results in simplified code, greater overall software quality and shorter
development times.

From Perl module developer perspective, Ext.Direct is just a transport layer;
it doesn't matter if a method is called from Perl code or through
Ext.Direct. This approach, in particular, allows for multi-tiered testing:

=over 4 

=item *

Server side methods can be tested without setting up HTTP environment
with the usual tools like L<Test::More>

=item *

Server side classes can be tested as a whole via Ext.Direct calls
using L<Test::ExtDirect>

=item *

Major application components can be tested with browser automation tools
like Selenium.

=back

=head1 ARCHITECTURE OVERVIEW

Ext.Direct is not only a transport protocol; in fact it implies a set
of requirements for the server side to meet. There are many web server
environments available in the Perl world, with several major interface
conventions established over the years. These environments are common
in the way that all of them implement HTTP request/response model;
however the particular details can differ significantly.

To deal with the web server differences, RPC::ExtDirect adopted the
core-periphery architecture. Transport layer core, provided by the
L<RPC::ExtDirect> CPAN distribution, is complemented by a peripheral
distribution called a I<gateway> that works with a particular web
server environment. The gateway is responsible for implementing the
features missing in a web server interface, if any.

Since the gateway modules implement the "lowest common denominator"
abstraction layer, it is fairly easy to use an Ext.Direct application
with several different gateways at the same time. The most common use
for this is testing; the added benefit is that the application becomes
independent of the web server environment and can be ported easily if
such a need arises.

See L<RPC::ExtDirect/GATEWAYS> for a list of available gateways.

=head1 TERMINOLOGY

Ext.Direct uses the following terms, followed by their descriptions:

=over 4

=item API

Description of server side calls exposed to client side. API consists
of L<remoting|/"Remoting API"> and L<polling|/"Polling API"> parts.

=item API declaration

JavaScript chunk that encodes L</API>. Usually generated
by application server and retrieved by client once upon startup.
Another option is to embed API declaration in client side application
code.

API declaration is generated by L<RPC::ExtDirect::API> module.

=item Remoting API

The main part of the L</"API declaration">, it declares the L<Actions|/Action>
and L<Methods|/Method> available to the client, as well as their calling
patterns, and other parameters.

=item Polling API

Used to declare the existence of L<Event Providers|/"Event Provider"> and
their credentials, basically the URI to use.

=item Router

Server side component that receives remoting calls, dispatches requests,
collects and returns call L<Results|/Result> or L<Exceptions|/Exception>.

=item Action

Namespace unit; collection of L<Methods|/Method>. The nearest Perl analog
is a package, other languages may call it a Class.

Since Ext.Direct originated in JavaScript, '::' will be replaced with dots
for all Actions in the L</"API declaration">, and should be called as
C<Action.Method()> instead of Perl style C<Action::Method>.

=item Method

Subroutine exposed through Ext.Direct API to be called by client side.
Method is fully qualified by L</Action> and Method names using a dot as
the delimiter: C<Action.Method>.

=item Method stub

JavaScript function created by the Ext.Direct transport layer on the client
side I<in lieu> of the actual L</Method> that only exists on the server
side. A separate stub will be created for each Method, with the parameter
signature conforming to Method's declaration in the L</API>.

=item Ordered Method

A L</Method> that accepts zero or more parameters in ordered fashion, or by
position (in a list). See more in
L<RPC::ExtDirect/"METHODS AND CALLING CONVENTIONS">.

=item Named Method

A L</Method> that accepts parameters by name, in a hash. See more in
L<RPC::ExtDirect/"METHODS AND CALLING CONVENTIONS">.

=item Form Handler Method

A L</Method> that accepts form submits. All form field values are passed to
the Form handler in a hash, C<field => value>. The only practical reason to
use Form handlers is to process file uploads; see
L<FILE UPLOADS|RPC::ExtDirect/"FILE UPLOADS"> for more information. See also
L<RPC::ExtDirect/"METHODS AND CALLING CONVENTIONS"> for more information on
Form handlers calling convention.

=item Poll Handler Method

A L</Method> that is called by an L</"Event Provider"> to return the list of
L<Events|/Event> to be passed on to the client side. See more in
L<RPC::ExtDirect/"METHODS AND CALLING CONVENTIONS">.

=item Result

Any data returned by a L</Method> upon successful or unsuccessful call
completion. This includes application logic errors. 'Not authenticated'
and alike events should be returned as Results, not L<Exceptions|/Exception>.

=item Exception

Fatal error, or any other unrecoverable event in the application code.
Calls that produce Exception instead of L</Result> are considered
unsuccessful; Ext.Direct provides built in mechanism for managing
Exceptions.

Exceptions are not used to indicate errors in application logic flow,
only for catastrophic conditions. Nearest analog is status code 500
for HTTP responses.

Examples of Exceptions are: request JSON is broken and can't be decoded;
called Method dies because of internall error; Result cannot be encoded
in JSON, etc.

=item Event

An asynchronous notification that can be generated by server side and
passed to client side, resulting in some reaction. Events are useful
for status updates, progress indicators and other predictably occuring
conditions and events.

=item Event Provider

Server side script that gets polled by client side every C<n> seconds;
default C<n> is 3 but it can be changed in client side configuration.

=back

=head1 GETTING STARTED

The first step is to install and configure a
L<gateway|RPC::ExtDirect/GATEWAYS> that works with your chosen Web
server environment. Please refer to gateways' documentation for that.
It is recommended to install L<Test::ExtDirect> module as well, so
that you could write tests for your Ext.Direct code right from the
start.

When you have the gateway configured, it's time to publish some code
in the Ext.Direct L</API>. The easiest way to do this is to use the
C<ExtDirect> attribute with the subroutines that need to be published:

    package MyApp::Math;
    
    use RPC::ExtDirect Action => 'MyMath';
    
    sub add : ExtDirect(len => 2) {
        my ($class, $a, $b) = @_;
        
        return $a + $b;
    }

In this snippet, we have published the C<MyApp::Math> package as
an Ext.Direct L</Action> called C<MyMath>, and added one L</Method>
to be exposed to the client side as C<MyMath.add>. In your Ext JS
or Sencha Touch app, use the Method with an asynchronous callback
function that will be fired when result is transmitted back to the
browser:

    MyMath.add(a, b, function(result) {
        alert('Multiplication result: ' + result);
    });

=head1 TESTING YOUR CODE

It is always a good idea to cover your code with unit tests; even
more so for something as complex as Remote Procedure Call APIs.
A lot of things can go wrong and break the server interface you
provide to JavaScript applications. What is worse, these things
rarely go wrong I<upfront>, usually the breakage creeps in gradually
over time. The only way to ensure that your server side API keeps
working exactly as the client side expects it to is via continuous
testing.

However, testing a server side API with the actual JavaScript code
that will consume the API is a daunting task. You would need an
instance of the Web server you are going to use in production, a
headless Web browser, a ton of infrastructure to keep all this up
to date; not even mentioning the time spent maintaining the whole
setup. If this picture makes you cringe with frustration, that's
totally understandable. But fear not, RPC::ExtDirect has a truly
Perlish answer to this question (making hard things possible).

L<Test::ExtDirect> is the recommended way to unit test your Ext.Direct
code. To be precise, it is hardly right to call it unit testing when
it involves a test HTTP server, a Perl client that makes actual RPC
invocations over HTTP and simulates the JavaScript client; it may
be more correct to refer to this process as integration testing instead.
However RPC::ExtDirect tries to be completely transparent and never
get in the way, so you can think of it as "testing functional units
of my code, disregarding the transport".

Supposed that you are sold on the idea, let's see how a unit test
for the C<add> subroutine created above looks in practice:

    # 01_math.t
    
    use Test::More tests => 1;
    use Test::ExtDirect;
    
    use MyApp::Math;
    
    my $result = call_extdirect(
        action => 'MyMath',
        method => 'add',
        arg    => [ 2, 2 ],
    );
    
    is $result, 4, "addition result matches";

Now, how hard is that?!

=head1 GOING FURTHER

Now that we have covered the basics, let's see what else RPC::ExtDirect
has in store:

=head2 Named parameters

Using named parameters is as easy as ordered ones:

    sub named : ExtDirect(params => ['foo', 'bar']) {
        my ($class, %arg) = @_;
        
        my $foo = $arg{foo};
        my $bar = $arg{bar};
        
        # do something, return a scalar
        my $result = ...;
        
        return $result;
    }

By default, the C<named> method above will receive B<only> the parameters
declared in the C<ExtDirect> attribute. To accept all named parameters,
turn off strict checking:

    sub named_no_strict
        : ExtDirect(params => ['foo', 'bar'], strict => !1)
    {
        my ($class, %arg) = @_;
        
        my $foo = $arg{foo};
        my $bar = $arg{bar};
        my $baz = $arg{baz}; # this parameter is undeclared but gets there
        
        ...
    }

=head2 Form submits

An Ext.Direct Method can be used to accept form submits in
both C<application/x-www-form-urlencoded> and C<multipart/form-data>
encodings. This feature can be used to accept file uploads from
non-HTML5 browsers (think IE9 and below):

    sub handle_upload : ExtDirect(formHandler, upload_arg => 'files') {
        my ($class, %arg) = @_;
        
        my $files = $arg{files}; # arrayref of files
        
        for my $file ( @$files ) {
            ...
        }
    }

=head2 Handling errors

When your server side code encounters an irrecoverable error, it is
a Good Thing to let the client side application know about it. The
usual way is to throw an L</Exception>:

    sub dying : ExtDirect(len => 0) { # no parameters
        die "Houston, we've got a problem!\n";
    }

L<Ext.Direct specification|http://www.sencha.com/products/extjs/extdirect>
requires the server side stack to only send exceptions in debugging
mode but never in production mode. The implied security concerns are valid,
but having two sets of logic would be unwieldy; RPC::ExtDirect compromises
by sending generic exceptions "An error has occured" in production mode
(default).

To turn on global debugging, set the L<debug option|RPC::ExtDirect::Config/debug>
in the L<global API instance's|RPC::ExtDirect::API/"GLOBAL API TREE INSTANCE">
Config:

    # Place this in the main app server code
    use RPC::ExtDirect;
    
    RPC::ExtDirect->get_api->config->debug(1);

If you are comfortable with exposing internal details of your app server to
the outside world even in production mode, turn on
L<verbose_exceptions|RPC::ExtDirect::Config/verbose_exceptions>:

    # This goes to the main app server, too
    use RPC::ExtDirect;
    
    RPC::ExtDirect->get_api->config->verbose_exceptions(1);

=head2 Using environment objects

Suppose that you want to restrict some parts of the API to be accessible
only by authenticated users. The usual way to do this is by using HTTP
cookies. However, cookies are not exposed to every Ext.Direct Method
by default; you need to tell RPC::ExtDirect to pass an
L<environment object|RPC::ExtDirect/"ENVIRONMENT OBJECTS"> to your
Method in order to get low level things like cookies or HTTP request
headers:

    sub restricted : ExtDirect(params => ['foo'], env_arg => 'env') {
        my ($class, %arg) = @_;
        
        my $env = $arg{env}; # Get the env object
        
        my $user = $env->cookie('user');
        
        # \0 is a shortcut for JSON::false
        return { success => \0, error => 'Not authenticated' }
            unless $user eq 'foo';
        
        ...
    }

=head2 Using hooks

In the example above, we checked the user's loginedness in the Method
that is supposed to do some actual work. Duplicating that code for every
restricted Method would be very tedious and error prone; what if we
could handle such things in a centralized subroutine that would be
called before each restricted Method's invocation and cancel it if
the user is not authenticated?

That is what L<Hooks|RPC::ExtDirect/HOOKS> are for:

    sub check_user {
        my ($class, %arg) = @_;
        
        # Hooks always receive an env object
        my $env = $arg{env};
        
        my $user = $env->cookie('user');
        
        # This hashref will be returned to the client side
        # as if the actual Method returned it
        return { success => \0, error => 'Not authenticated' }
            unless $user eq 'foo';
        
        # 1 means we're good to go
        return 1;
    }
    
    # This Method won't even be called unless the user is logged in
    sub foo : ExtDirect(params => ['foo'], before => \&check_user) {
        my ($class, %arg) = @_;
        
        ...
    }
    
    # Same thing, and no code duplication
    sub bar : ExtDirect(len => 2, before => \&check_user) {
        my ($class, $a, $b) = @_;
        
        ...
    }

In the example above, individual Method level hooks were used; you can
also assign hooks to an L</Action> (package), or globally. Global hooks
are very useful for doing application-wide things like security audit
logging:

    package MyApp::SecurityLog;
    
    sub log_everything {
        my ($class, %arg) = @_;
        
        my ($method, $result, $ex) = @arg{ qw/ method result exception / };
        
        # Store this into database, or do something else
        ...
    }


    # This goes to the main program
    RPC::ExtDirect->get_api->after('MyApp::SecurityLog::log_everything');

Set this way, C<log_everything> will be called after every Ext.Direct
Method invocation, even if a C<before> hook canceled it. See
L<RPC::ExtDirect::API::Hook> for the gory detail.

=head1 SEE ALSO

Live code examples are provided with L<CGI::ExtDirect> module in the
C<examples> directory.

=cut