Net::Google::SafeBrowsing4 - Perl extension for the Google Safe Browsing v4 API.
use Log::Log4perl qw(:easy); use Net::Google::SafeBrowsing4; use Net::Google::SafeBrowsing4::Storage::File; Log::Log4perl->easy_init($DEBUG); my $storage = Net::Google::SafeBrowsing4::Storage::File->new(path => '.'); my $gsb = Net::Google::SafeBrowsing4->new( key => "my key", storage => $storage, logger => Log::Log4perl->get_logger(), ); $gsb->update(); my @matches = $gsb->lookup(url => 'http://ianfette.org/'); if (scalar(@matches) > 0) { print("http://ianfette.org/ is flagged as a dangerous site\n"); } $storage->close();
Net::Google::SafeBrowsing4 implements the Google Safe Browsing v4 API.
The Google Safe Browsing database must be stored and managed locally. Net::Google::SafeBrowsing4::Storage::File uses files as the storage back-end. Other storage mechanisms (databases, memory, etc.) can be added and used transparently with this module.
The source code is available on github at https://github.com/juliensobrier/Net-Google-SafeBrowsing4.
If you do not need to inspect more than 10,000 URLs a day, you can use Net::Google::SafeBrowsing4::Lookup with the Google Safe Browsing v4 Lookup API which does not require to store and maintain a local database.
IMPORTANT: Google Safe Browsing v4 requires an API key from Google: https://developers.google.com/safe-browsing/v4/get-started.
Several constants are exported by this module:
Google requested to reset (empty) the local database.
An internal error occurred.
The server sent an error back to the client.
No update was performed, probably because it is too early to make a new request to Google Safe Browsing.
No data was sent back by Google to the client, probably because the database is up to date.
The update operation was successful.
Create a Net::Google::SafeBrowsing4 object
my $gsb = Net::Google::SafeBrowsing4->new( key => "my key", storage => Net::Google::SafeBrowsing4::Storage::File->new(path => '.'), lists => ["*/ANY_PLATFORM/URL"], );
Arguments
Safe Browsing base URL. https://safebrowsing.googleapis.com by default
Required. Your Google Safe Browsing API key
Required. Object which handles the storage for the Google Safe Browsing database. See Net::Google::SafeBrowsing4::Storage for more details.
Optional. The Google Safe Browsing lists to handle. By default, handles all lists.
Optional. Log::Log4perl compatible object reference. By default this option is unset, making Net::Google::SafeBrowsing4 silent.
Optional. Set to 1 to enable performance information logging. Needs a logger, performance information will be logged on DEBUG level.
Optional. Google Safe Browsing version. 4 by default
Optional. LWP::UserAgent to use for HTTPS requests. Use this option for advanced networking options, like proxies or local addresses.
Optional. Network timeout setting for LWP::UserAgent (60 seconds by default)
Optional. List of accepted compressions for HTTP response. Enabling all supported compressions reported by HTTP::Message by default.
Optional. maximum number of full hashes to request. (500 by default)
Performs a database update.
$gsb->update();
Returns the status of the update (see the list of constants above): INTERNAL_ERROR, SERVER_ERROR, NO_UPDATE, NO_DATA or SUCCESSFUL
This function can handle multiple lists at the same time. If one of the lists should not be updated, it will automatically skip it and update the other one. It is faster to update all lists at once rather than doing them one by one.
Optional. Update specific lists. Use the list(s) from new() by default. List are in the format "MALWARE/WINDOWS/URLS" or "*/WINDOWS/*" where * means all possible values.
Optional. Force the update (1). Disabled by default (0).
Be careful if you set this option to 1 as too frequent updates might result in the blacklisting of your API key.
Gets all threat list names from Google Safe Browsing and save them.
my $lists = $gsb->get_lists();
Returns an array reference of all the lists:
[ { 'threatEntryType' => 'URL', 'threatType' => 'MALWARE', 'platformType' => 'ANY_PLATFORM' }, { 'threatEntryType' => 'URL', 'threatType' => 'MALWARE', 'platformType' => 'WINDOWS' }, ... ] or C<undef> on error. This method updates C<$gsb->{last_error}> field.
Looks up URL(s) against the Google Safe Browsing database.
Returns the list of hashes along with the list and any metadata that matches the URL(s):
( { 'lookup_url' => '...', 'hash' => '...', 'metadata' => { 'malware_threat_type' => 'DISTRIBUTION' }, 'list' => { 'threatEntryType' => 'URL', 'threatType' => 'MALWARE', 'platformType' => 'ANY_PLATFORM' }, 'cache' => '300s' }, ... )
Optional. Lookup against specific lists. Use the list(s) from new() by default.
Required. URL to lookup.
These functions are not intended to be used externally.
Transforms a list from a string expression (eg.: "MALWARE/*/*") into a list object.
Handle server errors during a database update.
Formats the list objects for update requests.
Requests full full hashes for specific prefixes from Google.
Processes the request for full hashes from Google.
To use a proxy or select the network interface to use, simply create and set up an LWP::UserAgent object and pass it to the constructor:
use LWP::UserAgent; use Net::Google::SafeBrowsing4; use Net::Google::SafeBrowsing4::Storage::File; my $ua = LWP::UserAgent->new(); $ua->env_proxy(); # $ua->local_address("192.168.0.14"); my $gsb = Net::Google::SafeBrowsing4->new( key => "my-api-key", storage => Net::Google::SafeBrowsing4::Storage::File->new(path => "."), http_agent => $ua, );
Note that the Net::Google::SafeBrowsing4 object will override certain LWP properties:
The network timeout will be set according to the http_timeout constructor parameter.
http_timeout
The Content-Type default header will be set to application/json for HTTPS Requests.
The Accept-Encoding default header will be set according to the http_compression constructor parameter.
http_compression
See Net::Google::SafeBrowsing4::URI about URI parsing for Google Safe Browsing v4.
See Net::Google::SafeBrowsing4::Storage for the list of public functions.
See Net::Google::SafeBrowsing4::Storage::File for a back-end storage using files.
Google Safe Browsing v4 API: https://developers.google.com/safe-browsing/v4/
Julien Sobrier, <julien@sobrier.net>
Copyright (C) 2017 by Julien Sobrier
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.8 or, at your option, any later version of Perl 5 you may have available.
To install Net::Google::SafeBrowsing4, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Net::Google::SafeBrowsing4
CPAN shell
perl -MCPAN -e shell install Net::Google::SafeBrowsing4
For more information on module installation, please visit the detailed CPAN module installation guide.