Can I use wget to check , but not download

wget
mike628 picture mike628 · Jun 7, 2011 · Viewed 100.7k times · Source

Can I use wget to check for a 404 and not actually download the resource? If so how? Thanks

Answer

Shadikka picture Shadikka · Jun 7, 2011

There is the command line parameter --spider exactly for this. In this mode, wget does not download the files and its return value is zero if the resource was found and non-zero if it was not found. Try this (in your favorite shell):

wget -q --spider address
echo $?

Or if you want full output, leave the -q off, so just wget --spider address. -nv shows some output, but not as much as the default.

Related questions

Using wget to recursively fetch a directory with arbitrary files in it

I have a web directory where I store some config files. I'd like to use wget to pull those files down and maintain their current structure. For instance, the remote directory looks like: http://mysite.com/configs/.vim/ .vim holds …

Read More
How to set proxy for wget?

I want to download something with wget using a proxy: HTTP Proxy: 127.0.0.1 Port: 8080 The proxy does not need username and password. How can I do this?

Read More
How to specify the location with wget?

I need files to be downloaded to /tmp/cron_test/. My wget code is wget --random-wait -r -p -nd -e robots=off -A".pdf" -U mozilla http://math.stanford.edu/undergrad/ So is there some parameter to specify the directory?

Read More