webmirror - Simple WWW mirror program
webmirror [--from URL] [--to directory] [--proxy] [--checklinks] [--verbose]
This program can be used to mirror parts of WWW servers to a local directory. The traversal of the remote pages is recursive, that is, linked pages are mirrored, too, if they start with the same prefix. For example, if you mirror
http://www.math.fu-berlin.de/~leitner/perl/webmirror would be mirrored, too, but
http://www.math.fu-berlin.de/~leitner/ would not. These are the meanings of the options:
Where to mirror from. This has to be an HTTP URL.
Where to put the mirrored files.
Use the WWW proxy settings from the environment (http_proxy, no_proxy). You should not use this if your proxy is a caching proxy, only if it is a firewall proxy. Otherwise you may mirror old versions of the pages.
Check not only links that have the same prefix but one level of other links, too. This is very useful for checking your own pages for invalid outbound links.
Print debugging information.
webmirror normally prints only errors, for example:
Getting http://www.math.fu-berlin.de/~leitner/mutt/idnex.html Error: 404 not found Linked from: http://www.math.fu-berlin.de/~leitner/index.html
webmirror is Copyright (c) 1996 Felix von Leitner. All rights reserved. libwww-perl is Copyright (c) 1995, 1996 Gisle Aas. All rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.