Delays between requests in wget

wget
Snochacz picture Snochacz · Mar 3, 2016 · Viewed 13.6k times · Source

I want to download web using wget, but to make it more like a real user I would like to make small random delays between requests. I'm executing wget via cmd.

Answer

A.A picture A.A · Mar 5, 2016

You can add the below code into your command line which adds a ten second wait in between server requests.

-w 10

And you can also include

--random-wait

Into your command line with -w option which will vary the wait by 0.5 and 1.5 times the value you provide here.

Related questions

Using wget to recursively fetch a directory with arbitrary files in it

I have a web directory where I store some config files. I'd like to use wget to pull those files down and maintain their current structure. For instance, the remote directory looks like: http://mysite.com/configs/.vim/ .vim holds …

Read More
How to set proxy for wget?

I want to download something with wget using a proxy: HTTP Proxy: 127.0.0.1 Port: 8080 The proxy does not need username and password. How can I do this?

Read More
How to specify the location with wget?

I need files to be downloaded to /tmp/cron_test/. My wget code is wget --random-wait -r -p -nd -e robots=off -A".pdf" -U mozilla http://math.stanford.edu/undergrad/ So is there some parameter to specify the directory?

Read More