On Feb 5, 12:22, Lawrence LeMay wrote:
See above. CD burners are wonderful.
Somewhat out of question, but does anyone know of any software that would
download a web site, or a portion, to a local disk? IE, I would want to
be
able to archive part of a web site onto a CDrom,
without having to
manually
save every image and web page, and manually edit the
html links, etc.
Ideally the software would understand that I was planning to burn
650 meg, or 700 meg, etc images, and would break the download into
separate directories/images.
wget on a unix or Linux system. You'd want to pick your root point and
start wget with the -r (--recursive) and -k (or --convert-links) options
(to make any absolute links into relative ones, wherever possible) and the
-np (or --no-parent) so it doesn't follow links that go *up* the directory
tree rather than down (a good way to get lots more than you intended!).
There's also a -m (--mirror) option. You can refine things with other
options (such as those to avoid certain files or types, useful to exclude
those extra ?D=A indexes from directory listings) but I usually start with
-r -k -np. On a Windows system? Telnet to a Unix machine. There is a
Windows port, but I'm not sure how good it is.
wget can limit the total size of a download by setting a quota, but it's
not as useful as you might think. Better for you to decide yourself where
to split a collection over two or more CDs. Of course some sites make this
easy (like PUPS/TUHS, for which I run one of the official mirrors).
Another polite option is --wait, which waits between fetches to reduce the
bandwidth. Some sites might not like you hogging their connection. In
fact, recently I came across another site that had banned certain addresses
altogether for attempting to download a whole site without asking first.
The owner took the view that no-one could possibly want the whole site,
unless for a mirror, in which case it's best to ask.
Some sites have a megabytes-per-month limit imposed by their ISP, so again,
ask.
There are also programs for FTP (only) mirroring: the imaginatively-named
'mirror' and 'ftpmirror' come to mind, and pavuk too, but wget is a
pretty
good all-rounder.
http://www.wget.org/
http://www.gnu.org/software/wget/wget.html
http://www.idata.sk/~ondrej/pavuk/
http://noc.intec.co.jp/ftpmirror.html
http://sunsite.org.uk/packages/mirror/
--
Pete Peter Turnbull
Network Manager
University of York