<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<link href="http://sadiframework.org/style/sadi.css" type="text/css" rel="stylesheet"/>
<style type="text/css">
h2 {
margin-top: 2em;
margin-left: 2em
}
h3 {
margin-top: 2em;
margin-left: 4em
}
h4 {
margin-top: 2em;
margin-left: 6em
}
p {
margin-left: 6em
}
pre {
margin-left: 8em;
margin-right: 8em;
padding: 4px;
font-family: monospace;
font-size: 125%;
background: #eee;
}
</style>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>Building an Asynchronous SADI Service (Advanced)</title>
</head>
<body>
<h1>Building an Asynchronous SADI Service With Perl</h1>
<p>
This document summarises steps needed in order to develop (and to implement) an asynchronous
SADI web service using SADISeS (<strong>SADI</strong> <strong>S</strong>ervice
<strong>S</strong>upoort).</p>
<p>The main thing to understand is that SADISeS does not give you a full implementation of your service.
You still need to program the business logic (e.g. to extract data from your database) -
but you do not need to worry about the SADI or HTTP protocol details. </p>
<p>At the end of this tutorial, you should have 3 things: a SADI service definition file, a PERL module containing your business logic, and a PERL CGI entry script to your service.</p>
<p>The service definition file is a properties based file that contains the information needed to describe what a SADI service will do. In most cases, this file will be located in the <em><strong>~/Perl-SADI/definitions</strong></em> directory* and can be manually edited to reflect what your SADI service does.</p>
<p>The SADISeS generated PERL module contains a place for you to insert the business logic of your service. In most cases, this file will be located in the <em><strong>~/Perl-SADI/services/Service/</strong></em> directory*.</p>
<p>Finally, the PERL CGI entry script provides the interface with which a user interacts with your SADI service. This script is also generated by SADISeS and is located in the <em><strong>~/Perl-SADI/cgi</strong></em> directory*.</p>
<p class="query-english"><strong><font color="#336633">*For those of you developing your SADI services using MS Windows, the<em> ~</em> refers to your home directory. On Windows, this is usually <em>C:\Users\Your_Name.</em></font></strong></p>
<p>These 3 things are all you need to create a single SADI web service and SADISeS helps you do just that without worrying too much about the SADI protocol. </p>
<p>Let's now move on towards building our SADI service. In step 1 below, we will make sure that you have all of the dependencies in order.</p>
<h2>Table of Contents</h2>
<p>
<a href="#step_1">Step 1: What is needed</a>
<br/>
<a href="#step_2">Step 2: Service definition generation</a>
<br/>
<a href="#step_3">Step 3: Service generation</a>
<br/>
<a href="#step_4">Step 4: Service implementation</a>
<br/>
<a href="#step_5">Step 5: Service testing</a>
<br/>
<a href="#step_6">Step 6: Service deployment</a>
<br/>
<a href="#step_7">Step 7: Service testing using HTTP</a>
<br/>
<a href="#step_8">Step 8: Service registration</a>
<br/>
</p>
<h3><a name="step_1"/>Step 1: What is needed</h3>
<p>To implement SADI services using SADISeS, you need to have the following installed on your machine:</p>
<ol>
<ol>
<ol>
<li>
<strong>Perl</strong> - perl has to be installed on your machine </li>
<li>
<strong>A web server</strong> - this document assumes that you are using Apache2 </li>
<li>
<strong>Perl SADI</strong> - available on <a href="http://search.cpan.org/dist/SADI/" target="_blank">cpan</a></li>
</ol>
</ol>
</ol>
<p>To implement this particular service, you will need in addition to the above requirements,</p>
<ol>
<ol>
<ol>
<li>DBI - available on cpan or your favorite perl package manager</li>
<li> DBD::mysql - also available on cpan or your favorite perl package manager</li>
</ol>
</ol>
</ol>
<p>
Once you have installed Perl SADI and all of its dependencies on your machine, you will have to run through the SADISeS set up. This is only done once per user of SADISeS (unless you are upgrading Perl-SADISeS). For more information, please view the SADI::SADI documentation.</p>
<h3><a name="step_2"/>Step 2: Service definition generation</h3>
<p>Before we can generate any code, we need to tell SADISeS a little bit about our service. This is done via a definitions file.</p>
<p>The service that we are going to implement in this document is one that given a GO record, will return the GO term name and GO term definition.</p>
<p>To generate a definition file for your service, issue the following command at the command prompt:</p>
<pre>$ sadi-generate-services.pl -D getGOTermAsync</pre>
<p>Basically, this tells SADISeS that we would like to generate a definition file for the service 'getGOTermAsync'. The generated file can be found in ~/Perl-SADI/definitions/getGOTermAsync.</p>
<p>If you open the generated definitions file, you will see something like the following:</p>
<pre># leave the following line as is!
ServiceName = getGOTermAsync
# modify the values below as you see fit.
ServiceType = http://someontology.org/services/sometype
InputClass = http://someontology.org/datatypes\#Input1
OutputClass = http://someontology.org/datatypes\#Output1
Description = A implementation of the 'getGOTermAsync' service
UniqueIdentifier = urn:lsid:myservices:getGOTermAsync
Authority = authority.for.getGOTermAsync
Authoritative = 1
Provider = myaddress@organization.org
ServiceURI = http://localhost/cgi-bin/getGOTermAsync
URL = http://localhost/cgi-bin/getGOTermAsync
SignatureURL = http://localhost/cgi-bin/getGOTermAsync</pre>
<p>SADISeS has done a nice job in preparing this file for us! </p>
<p><strong>Beware, if you use the characters # or = you will need to escape them with a \.</strong></p>
<p>We will have to edit it slightly, to ensure that we specify our actual inputs/outputs. For now, we will leave the URL as is.</p>
<p>Please specify that the <em><strong>InputClass</strong></em> is<strong> http://purl.oclc.org/SADI/LSRN/GO_Record</strong> and the <em><strong>OutputClass</strong></em> is <strong>http://sadiframework.org/ontologies/service_objects.owl#getGOTermAsync_Output</strong>. Save and close the file!</p>
<p>The edit file is shown below for clarity:</p>
<pre># leave the following line as is!
ServiceName = getGOTermAsync
# modify the values below as you see fit.
ServiceType = http://someontology.org/services/sometype
<strong>InputClass = http://purl.oclc.org/SADI/LSRN/GO_Record
OutputClass = http://sadiframework.org/ontologies/service_objects.owl\#getGOTermAsync_Output</strong>
Description = A implementation of the 'getGOTermAsync' service
UniqueIdentifier = urn:lsid:myservices:getGOTermAsync
Authority = authority.for.getGOTermAsync
Authoritative = 1
Provider = myaddress@organization.org
ServiceURI = http://localhost/cgi-bin/getGOTermAsync
URL = http://localhost/cgi-bin/getGOTermAsync
SignatureURL = http://localhost/cgi-bin/getGOTermAsync</pre>
<h3><a name="step_3"/>Step 3: Service generation</h3>
<p>Now that we have a definitions file, the next step in building a SADI service is to generate the actual service code!</p>
<p>To generate our service skeleton, issue the following command at the command prompt:</p>
<pre>$ sadi-generate-services.pl -A getGOTermAsync</pre>
<p>SADISeS will then go ahead and generate 2 things for you! An entry script for your service (located in the ~/Perl-SADI/cgi/ directory) and a service implementation file (located in the ~/Perl-SADI/services/Service/ directory).</p>
<p>Your entry script will be called getGOTermAsync. The implementation file is called getGOTermAsync.pm. Go ahead and look at both files. In the next section, we will be editing getGOTermAsync.pm.</p>
<h3><a name="step_4"/>Step 4: Service implementation</h3>
<p>Now that we are ready to implement the business logic, we will have to find, open and edit the module getGOTermAsync.pm (look in the folder <span class="query-english">~/Perl-SADI/services/Service/</span>).</p>
<p>SADISeS automatically created this file for you and left just the subroutine <span class="query-english"><strong>process_it</strong></span> for you to code your implementation. Fortunately, SADISeS provides some sample code for you to see how some operations are done!</p>
<p>One of the very first things that you will see in process_it is the line:</p>
<pre> foreach my $input (@inputs) { ...</pre>
<p>Basically, our service is iterating over any and all inputs recieved that are of class <em>InputClass</em>. It is then up to us to use that data in our business logic and output the result as class OutputClass.</p>
<p>The input, $input, is of type <a href="http://search.cpan.org/perldoc?RDF::Core::Resource" target="_blank">RDF::Core::Resource</a>.</p>
<p>Our business logic will be placed after the line:</p>
<pre># do something with $input ... (sorry, can't help with that)</pre>
<h4>Our business logic:</h4>
<p>Before doing anything, we need to add the following use statements:</p>
<pre>use DBI;<br>use DBD::mysql;</pre>
<p>Reading the inputs:</p>
<pre>my $accession = $input->getURI;
# get just the id number
$accession = $1 if $accession =~ /(\d+)$/;
# prefix our accession with GO
$accession = "'GO:$accession'";</pre>
<p>Performing our calculation on the data:</p>
<pre># before doing anything, we will allow our script to sleep for 15 seconds
# this mimics a long running service
sleep(15);
# open a db connection
my ($dsn) = "DBI:mysql:go_latest:mysql.ebi.ac.uk:4085";
my $dbh = DBI->connect($dsn, 'go_select', 'amigo', {RaiseError => 1})
or $self->throw( "getGOTermAsync: can't connect to database" );
# throw an exception unless $dbh ...
$self->throw("getGOTermAsync: problem with dbh!") unless $dbh;
# here we debug our select statement -> to our SADI log file
$LOG->debug ("select acc, name, term_definition from term, term_definition where term.id = term_definition.term_id and acc IN ($accession)");
my $sth = $dbh->prepare("select acc, name, term_definition from term, term_definition where term.id = term_definition.term_id and acc IN ($accession)");
$sth->execute;
</pre>
<p>Populating our output:</p>
<pre>while (my ($acc, $name, $def) = $sth->fetchrow_array()){
my $id = ($acc =~ /GO:(\d+)/ && $1); # need to strip-off the GO: prefixs
$core->addOutputData(
node => $input,<br> value => $name,
predicate => "http://sadiframework.org/ontologies/predicates.owl#hasTermName"
);
$core->addOutputData(
node => $input,
value => $def,
predicate => "http://sadiframework.org/ontologies/predicates.owl#hasTermDefinition"
);
}
# lets sleep again for 5 seconds
sleep(5);</pre>
<p>Notice how we call addOutputData() each time we wish to add something to our output document. Save and close the file. We will test our service in the next section.</p>
<h3><a name="step_5"/>Step 5: Service testing</h3>
<p>
Now that we have implemented our service, we will test it to make sure that it works. The input that we will be using is shown below:
</p>
<pre><rdf:RDF
xmlns="http://www.w3.org/2002/07/owl#"
xml:base="http://bioinfo.icapture.ubc.ca/SADI"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:lsrn="http://purl.oclc.org/SADI/LSRN/"
xmlns:owl="http://www.w3.org/2002/07/owl#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<lsrn:GO_Record rdf:about="http://lsrn.org/GO:0043116"/>
</rdf:RDF></pre>
<p>
Copy and save the input to a file (I will assume that the file is saved as go-input.xml).
</p>
<p>
Assuming that you saved the file under the name go-input.xml, our SADI service can be tested with the following command:
</p>
<pre>$ sadi-testing-service.pl Service::getGOTermAsync go-input.xml</pre>
<p>
The expected output for our service:
</p>
<pre><rdf:RDF
xmlns:a="http://sadiframework.org/ontologies/predicates.owl#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>
<rdf:Description rdf:about="http://lsrn.org/GO:0043116">
<a:hasTermName>negative regulation of vascular permeability</a:hasTermName>
<a:hasTermDefinition>Any process that reduces the extent to which blood vessels can be pervaded by fluid.</a:hasTermDefinition>
</rdf:Description>
</rdf:RDF></pre>
<p>
If you have no errors, then proceed to step 6! If you have some errors, hopefully SADISeS stack trace will help you pinpoint the problem!
</p>
<h3><a name="step_6"/>Step 6: Service deployment</h3>
<p>Deploying our SADI service is extremely straight forward!</p>
<p>The only thing you need to do is to tell your Web Server where the cgi script that we generated is located.</p>
<p>If you recall, our services' cgi script was called getGOTermAsync (one of the files generated using sadi-generate-services.pl)</p>
<p> Make a symbolic link from the cgi-bin directory of your Web Server (e.g on some Linux distributions, using Apache Web server, the cgi-bin directory is /usr/lib/cgi-bin) to the cgi-bin script.</p>
<p>For example:</p>
<pre>cd /usr/lib/cgi-bin
sudo ln -s /home/ekawas/Perl-SADI/cgi/getGOTermAsync . </pre>
<p>Every time that you generate a cgi service using Perl SADI, you will have to perform an operation similar to this one for the service that you created in order to deploy it.</p>
<h3><a name="step_7"/>Step 7: Service testing using HTTP</h3>
<p>Now that the service has been deployed, you can test it using HTTP. The input that we will be using is the same as for local testing of our service:</p>
<pre><rdf:RDF
xmlns="http://www.w3.org/2002/07/owl#"
xml:base="http://bioinfo.icapture.ubc.ca/SADI"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:lsrn="http://purl.oclc.org/SADI/LSRN/"
xmlns:owl="http://www.w3.org/2002/07/owl#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<lsrn:GO_Record rdf:about="http://lsrn.org/GO:0043116"/>
</rdf:RDF> </pre>
<p>Copy and save the input to a file. </p>
<p>Assuming that you saved the file under the name regression-input.xml, our SADI service can be tested with the following command:</p>
<pre>$ sadi-testing-service.pl -e http://localhost/cgi-bin/getGOTermAsync go-input.xml</pre>
<p>Of course, you may need to modify the URL http://localhost/cgi-bin/getGOTermAsync (to the actual address that you deployed the service to!).</p>
<p>When we call the script with the -e option, we tell the sadi-testing-service.pl script that we would like to call our service using HTTP Post. We then must provide the script with 1 (or an optional second) parameter:</p>
<ol>
<ol>
<ol>
<li>the url to the service</li>
<li>an optional file containing the input to our service</li>
</ol>
</ol>
</ol>
<p>The expected output should is a bit different than what we saw in local testing:</p>
<pre>HTTP/1.1 200 OK
Connection: close
Date: Fri, 18 Sep 2009 16:16:58 GMT
Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.3 with Suhosin-Patch
Vary: Accept-Encoding
Content-Type: text/xml; charset=ISO-8859-1
Client-Date: Fri, 18 Sep 2009 16:16:59 GMT
Client-Peer: 127.0.0.1:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
<rdf:RDF
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:j.0="http://sadiframework.org/examples/regression.owl#" >
<rdf:Description rdf:about="http://purl.oclc.org/SADI/LSRN/GO_Record">
<rdf:type rdf:resource="http://sadiframework.org/ontologies/service_objects.owl#getGOTerm_Output"/>
<rdfs:isDefinedBy rdf:resource="http://localhost/cgi-bin/getGOTermAsync?poll=r6j0e91j6LvQ9JkN21uEaPjk"/>
</rdf:Description>
</rdf:RDF> </pre>
<p>As this is an asynchronous service, the service output tells you that the output isDefinedBy http://localhost/cgi-bin/getGOTermAsync?poll=r6j0e91j6LvQ9JkN21uEaPjk. This URL is our polling URL.</p>
<p>So if we issue the following command (notice the <em>-g</em> parameter for performing an <em><strong>HTTP GET</strong></em>):</p>
<pre>$ sadi-testing-service.pl -g http://localhost/cgi-bin/getGOTermAsync?poll=r6j0e91j6LvQ9JkN21uEaPjk</pre>
<p>One of 2 things will happen:</p>
<ol>
<ol>
<ol>
<li>The service will send you a HTTP redirect and set the sadi-please-wait header. This header will outline how long, in milliseconds, you should wait before polling again.</li>
<li>The service will send you the output for your invocation, since you waited for service execution to finish.</li>
</ol>
</ol>
</ol>
<p>Assuming that you issued the above command right after issuing the intial request, the output from the testing script will look something like:</p>
<pre>HTTP/1.1 302 Found
Connection: close
Date: Fri, 18 Sep 2009 16:30:16 GMT
<strong>Pragma: sadi-please-wait = 120000</strong>
Location: http://localhost/cgi-bin/getGOTermAsync?poll=r6j0e91j6LvQ9JkN21uEaPjk
Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.3 with Suhosin-Patch
Content-Length: 0
Content-Type: text/plain
Client-Date: Fri, 18 Sep 2009 16:30:16 GMT
Client-Peer: 127.0.0.1:80
Client-Response-Num: 1
Client-Warning: Redirect loop detected (max_redirect = 7)
Done!</pre>
<p>We will wait a minute before re-issuing the HTTP GET on our polling URL. We should now see the service output:</p>
<pre>HTTP/1.1 200 OK
Connection: close
Date: Fri, 18 Sep 2009 17:20:11 GMT
Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.3 with Suhosin-Patch
Vary: Accept-Encoding
Content-Length: 427
Content-Type: text/xml; charset=ISO-8859-1
Client-Date: Fri, 18 Sep 2009 16:31:11 GMT
Client-Peer: 127.0.0.1:80
Client-Response-Num: 1
<rdf:RDF
xmlns:a="http://sadiframework.org/ontologies/predicates.owl#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
>
<rdf:Description rdf:about="http://lsrn.org/GO:0043116">
<a:hasTermName>negative regulation of vascular permeability</a:hasTermName>
<a:hasTermDefinition>Any process that reduces the extent to which blood vessels can be pervaded by fluid.</a:hasTermDefinition>
</rdf:Description>
</rdf:RDF>
Done!</pre>
<p>Our service has finished running and has returned to us the output data that we expected!</p>
<p><strong><em>To see what else the service testing script can do, run it without parameters or with the -h parameter.</em></strong></p>
<h3><a name="step_8"/>Step 8: Service registration</h3>
<p>Before we can register our service, we will need to open up the service definition file. Once this file is open, we need to verify a few things first!</p>
<p>First of all, we need to ensure that our URL/SignatureURL both point to the remote HTTP address of our entry script getGOTermAsync (during testing, this was http://localhost/cgi-bin/getGOTermAsync).</p>
<p>Second of all, we ... actually, there is no second of all! We just need to make sure that if we enter the remote HTTP address of our entry script getGOTermAsync in our web browser, we will see some XML (the SADI service signature) outputted.</p>
<p>To actually register our service, we need to open our browser to <a href="http://sadiframework.org/registry/" target="_blank">http://sadiframework.org/registry/</a> and enter our service URL into the textbox. Once we have done that, sadiframework.org will add our service to the list of services that it knows about!</p>
<p>That's all there is to constructing an asynchronous Perl SADI services! </p>
</body>
</html>