Apache::Gateway - A Bloated Gateway Module
Example Apache configuration:
<Location /CPAN/> SetHandler perl-script PerlHandler Apache::Gateway PerlSetVar GatewayConfig /etc/apache/gateway/CPAN PerlSetupEnv Off </Location>
Example GatewayConfig file:
GatewayRoot /CPAN/ <LocationMatch ".*"> Site ftp://ftp.perl.org/pub/perl/CPAN/ MuxSite ftp://ftp.cdrom.com/pub/perl/CPAN/ MuxSite ftp://ftp.digital.com/pub/plan/perl/CPAN/ Site ftp://ftp.orst.edu/pub/packages/CPAN/ Site ftp://ftp.funet.fi/pub/languages/perl/CPAN/ </LocationMatch> ClockBroken ftp://ftp.cdrom.com EET PST8PDT ClockBroken ftp://ftp.digital.com EET PST8PDT ClockBroken ftp://ftp.orst.edu EET PST8PDT ClockBroken ftp://ftp.perl.org CST CST6CDT
See the examples directory for commented examples.
Apache::Gateway module implements a gateway with assorted optional features.
Apache::Gateway services requests using
LWP and, hence, can gateway to any protocol that
LWP understands. It also makes foreign URIs appear to be local URIs.
Apache::Gateway does not include a cache, but it can be used in combination with a proxy cache to cache what the gateway retrieves. For example, Apache can provide caching for the gateway by setting up a proxy cache virtual host and a gateway virtual host and then using the proxy to access the gateway.
Multiple mirrors can provide an instance. Requests which fail will automatically be retried with the next mirror. This capability is very useful when some mirrors are busy or erratic.
Like the CPAN multiplexer,
Apache::Gateway can multiplex requests amongst several mirrors.
The origin server to contact can vary depending upon the URL. This capability is particularly useful when dealing with partial mirrors. A common situation is that some files may be available at all mirrors, but less commonly used files will only be available at a few mirrors.
(Need to think of a better name for this feature.) Remote FTP directory listings can be modified to refer to the gateway. This feature is somewhat similar to the ProxyPassReverse directive.
This feature was especially complicated and problematical. It has now been removed.
Apache::Gateway can try to correct incorrect timestamps generated by popular mirroring software. In particular, it can try to compensate for the way the Perl mirror program sets timestamps.
Most configuration is done in the GatewayConfig files. The regular Apache configuration files only need to include the handler directives and set the
GatewayConfig filename. Environment variables are not used, so
PerlSetupEnv can be Off.
GatewayConfig directives purposely look like Apache config directives so that the syntax will be familiar. However, GatewayConfig directives are not Apache config directives. They cannot be used in Apache config files (and vice versa)!
Sets the root of the gatewayed area on the local server. Generally matches the
Location setting in the Apache config files. Defaults to "/".
Passes timeout (in seconds) to
Begins a LocationMatch section. Works similarly to the ApacheLocation match directive except that the pattern is a Perl regexp. Note: there are no
Location or other style sections, only
LocationMatch sections are tried in order until a regexp is matched.
Sets an upstream server to contact for this URI. In case of failure, requests are automatically retried with successive sites in the order they appear. Failures can include anything from the upstream server being down or flaky to a file not being present because the upstream mirror is out of synch with its primary site.
Sets an upstream server to contact for this URI. Adjacent
MuxSite servers are tried in round robin order.
For example, here again is the default portion of the sample GatewayConfig file above:
<LocationMatch ".*"> Site ftp://ftp.perl.org/pub/perl/CPAN/ MuxSite ftp://ftp.cdrom.com/pub/perl/CPAN/ MuxSite ftp://ftp.digital.com/pub/plan/perl/CPAN/ Site ftp://ftp.orst.edu/pub/packages/CPAN/ Site ftp://ftp.funet.fi/pub/languages/perl/CPAN/ </LocationMatch>
MuxSite directives here, the first request will be forwarded to ftp.perl.org. If it fails, the request will be retried with cdrom, digital, orst, and funet, in that order. The next request for that process will be tried with ftp.perl.org first again. If it fails, retries then go to digital, cdrom, orst, and finally funet.
A good general strategy for packages with multiple mirrors might be to specify one or two nearby sites to try first. Then specify some multiplexed sites slightly further away in case the nearby sites fail. Finally, fall back to the primary site if all else fails.
When caching is employed and requests can be gatewayed to multiple mirrors, timestamp correctness becomes more important. Unfortunately, timestamps on mirrored files are usually wrong. For example, the popular Perl mirror program is generally configured to match timestamps using the local timezone both locally and on the server it is mirroring. This strategy is only guaranteed to work if both servers are in the same timezone.
Example: ClockBroken ftp://ftp.cdrom.com EET PST8PDT
cdrom gets files from funet, which seems always to use the EET timezone (which is two hours off from GMT) for purposes of mirroring. cdrom, however, uses the PST8PDT timezone, so that 00:00 on funet differs from 00:00 on cdrom by 9 or 10 hours, depending upon whether or not Daylight Savings Time is in effect. The example ClockBroken line corrects for this disparity.
Note: timezones are those understood by Time::Zone.
The following internal functions are documented (mostly useful for hackers):
Construct a new Apache::Gateway object describing a gateway. If a LWP::UserAgent is not provided, a new one will be created. Note: the user agent is modified for seach request; it is not constant and is probably not shareable.
Get/set the user agent.
Get/set the Apache request currently being gatewayed. To send the request, see the send_request method.
Get/set the configuration information for this gateway location. Can be overridden to provide dynamic per location information
Clear request headers in $r in preparation for a redirect.
Return semicanonicalized server URL (without trailing slash).
Return the (somewhat canonicalized) "server name" portion of the URL. The "server name" is defined as the leading scheme://authority portion of the URL.
Return the (somewhat canonicalized) "server name" portion of the URL of this server. The "server name" is defined as the leading scheme://authority portion of the URL. Currently assumes server access is via HTTP.
Get the usual time difference (in seconds) between the two time zones. Will yield the wrong results in the midst of a change to/from daylight savings time. Specifically, as used in this module, this function will return the wrong results when applied to files retrieved by the mirror during the two hours of the year when one server is in Daylight Savings Time and the other is not.
Update Via header in HTTP::Response with information about this hop. Hop information combines protocol information from the message with server information from the Apache server. The server name returned is hardcoded as '
Eventually, options should be provided to control hostname suppression and comment customization.
Copy the headers from an
HTTP::Headers object to an
Apache::Request. Hope that the Apache request object will later print out the headers in "Good Practice" order (there appears to be no way of controlling this).
The only tricky item is the Content-Type header, which needs special handling.
Try a redirect. We do this via
internal_redirect_handler does not provide hooks for detecting and recovering from errors.
Get/set the site tried. Can be used to determine which upstream server actually fields a request.
Try the site $gw->site. Ideally, we could use
Apache::internal_redirect_handler to try the redirects. However, it provides no hook for detecting an error and aborting output. That's not mod_perl's fault--Apache source would need to be modified to support such a hook.
Try sites in order until one succeeds. $allow_last_site_abort indicates if the last site can/should be aborted after examing the head for its error code. All other sites always allow premature abortion.
Abortion is needed because only one request can be allowed to run to completion and produce a message body.
Get the list of sites to try for this request. Can be overridden to customize the list of sites to try.
By default, this method looks through the LocationMatch sections in the GatewayConfig file in order and returns the sites in the first section matched.
Send the Apache request to the upstream server. Optionally sets it first.
Apache::Gateway is a big, complicated module that loads many other modules. As such, it pushes
mod_perl to its limits, especially when used with DSO/APXS.
The current version of
LWP (5.35) only supports If-Modified-Since for file and ftp URLs. Thus, gatewaying to ftp servers will actually be better than gatewaying to http servers for cached responses.
A ProxyRemote-like capability is needed for origin servers which must be accessed through a proxy.
A ProxyPassReverse analogue might be useful, too.
Apache::Gateway assumes it is being accessed using HTTP. Ought to handle cases where this gateway is accessed using https (SSL).
There is no way to tell LWP to use a proxy.
Server response header field should contain information about the origin server, not this server. Unfortunately, Apache overrides any existing origin server information in this field.
Charles C. Fu, email@example.com
perl(1), Apache(3pm), LWP(3pm).