How to get the contents of a webpage in a shell variable?

linux bash shell wget
Aillyn picture Aillyn · Sep 18, 2010 · Viewed 284k times · Source

In Linux how can I fetch an URL and get its contents in a variable in shell script?

Answer

codaddict picture codaddict · Sep 18, 2010

You can use wget command to download the page and read it into a variable as:

content=$(wget google.com -q -O -)
echo $content

We use the -O option of wget which allows us to specify the name of the file into which wget dumps the page contents. We specify - to get the dump onto standard output and collect that into the variable content. You can add the -q quiet option to turn off's wget output.

You can use the curl command for this aswell as:

content=$(curl -L google.com)
echo $content

We need to use the -L option as the page we are requesting might have moved. In which case we need to get the page from the new location. The -L or --location option helps us with this.

Related questions

How do I prompt for Yes/No/Cancel input in a Linux shell script?

I want to pause input in a shell script, and prompt the user for choices. The standard Yes, No, or Cancel type question. How do I accomplish this in a typical bash prompt?

Read More
How to run a shell script on a Unix console or Mac terminal?

I know it, forget it and relearn it again. Time to write it down.

Read More
How do I test if a variable is a number in Bash?

I just can't figure out how do I make sure an argument passed to my script is a number or not. All I want to do is something like this: test *isnumber* $1 && VAR=$1 || echo "need a number" Any …

Read More