Wubot::Plugin::WebFetch - fetch content from a URL
version 0.2_002
~/wubot/config/plugins/WebFetch/mypage.yaml --- delay: 24h url: http://myweb.com/somepage.html field: content
Fetch content from a web page at regular intervals.
The retrieved content will be stored in the 'field' specified in the config. If no field is specified, the default target field will be 'content'.
I like to keep track of how many hits some of my projects have received on github. I haven't found any sort of a feed that will provide that data. The only place I have seen that is on the 'traffic' graph page. So I use the following config to pull that page once per day, grab the number of hits and send me a notification.
--- delay: 24h url: https://github.com/wu/wubot/graphs/traffic react: - name: content condition: contains content rules: - name: get page views plugin: CaptureData config: source_field: content regexp: '^.*?Page Views \((\d+) over last 90 days\)' target_field: hits - name: build subject plugin: Template config: template: wubot: hits last 90 days: {$hits} target_field: subject - name: color plugin: SetField config: set: color: purple sticky: 1
My DSL router does not provide any mechanism to graph the traffic sent or received. However it does provide a web page where it lists the the total number of packets sent and received. So I use the following config which captures the sent and received packets every 5 minutes.
--- delay: 5m url: http://192.168.1.98/cgi-bin/webcm?getpage=../html/lan_status.html react: - name: content condition: contains content rules: - name: get packets sent plugin: CaptureData config: source_field: content regexp: '^.*?Packets Sent:</td>.*?<td\>(\d+)' target_field: sent - name: get packets received plugin: CaptureData config: source_field: content regexp: '^.*?Packets Received:</td>.*?<td>(\d+)' target_field: recv
Then I use the following rule in the reactor to graph the data using RRD:
- name: WebFetch RRD condition: key matches ^WebFetch-router plugin: RRD config: base_dir: /usr/home/wu/wubot/rrd fields: sent: COUNTER recv: COUNTER period: - day - week - month graph_options: lower_limit: 0 right-axis: 1:0 width: 375
This plugin uses Wubot::Util::WebFetcher.
To install Wubot, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Wubot
CPAN shell
perl -MCPAN -e shell install Wubot
For more information on module installation, please visit the detailed CPAN module installation guide.