How to download an entire directory and subdirectories using wget?

regex linux bash wget
code4fun picture code4fun · Jun 24, 2013 · Viewed 245.3k times · Source

I am trying to download the files for a project using wget, as the SVN server for that project isn't running anymore and I am only able to access the files through a browser. The base URLs for all the files is the same like

http://abc.tamu.edu/projects/tzivi/repository/revisions/2/raw/tzivi/*

How can I use wget (or any other similar tool) to download all the files in this repository, where the "tzivi" folder is the root folder and there are several files and sub-folders (upto 2 or 3 levels) under it?

Answer

user2936450 picture user2936450 · Oct 30, 2013

You may use this in shell:

wget -r --no-parent http://abc.tamu.edu/projects/tzivi/repository/revisions/2/raw/tzivi/

The Parameters are:

-r     //recursive Download

and 

--no-parent // Don´t download something from the parent directory

If you don't want to download the entire content, you may use: 

-l1 just download the directory (tzivi in your case)

-l2 download the directory and all level 1 subfolders ('tzivi/something' but not 'tivizi/somthing/foo')

And so on. If you insert no -l option, wget will use -l 5 automatically.

If you insert a -l 0 you´ll download the whole Internet, because wget will follow every link it finds.

Related questions

linux find regex

I'm having trouble using the regex of the find command. Probably something I don't understand about escaping on the command line. Why are these not the same? find -regex '.*[1234567890]' find -regex '.*[[:digit:]]' Bash, Ubuntu

Read More
Regular Expression in Bash Script

Hello awesome community, I'm a complete dope when it comes to regex. I've put off learning it.. and now my laziness has caught up with me. What I'm trying to do: Check if a string matches this format: 10_06_13 ie. Todays …

Read More
Recursively change file extensions in Bash

I want to recursively iterate through a directory and change the extension of all files of a certain extension, say .t1 to .t2. What is the bash command for doing this?

Read More