Skip to content
Tags

Recursive wget

by admin on October 14th, 2007

I love wget.  I’ve used it in the past to quickly snag the contents of entire web directories, and tonight I learned a few more tricks it can do.  Namely, it can recurse down as many levels deep from the first page as you like, it can ignore the reported length of items it is retrieving, it can automatically adjust all links in all pages it downloads to refer to each other correctly on the local archive, and it can run with nice progress indicators.  (This is by NO means the complete list of all the ways to configure it – these are just the ones I prefer to use.  Here is the set of switches I like:  (yes, they have more compact versions, but for clarity I’m using the longer forms.)
wget --recursive --progress=bar --ignore-length --level=1 --convert-links http://www.somedomain.com/page.html

No comments yet

Leave a Reply

Note: XHTML is allowed. Your email address will never be published.

Subscribe to this comment feed via RSS

Spam protection by WP Captcha-Free