Opening a public page from browser works fine.
Downloading same page using WebClient throws - (403) Forbidden.
What is going on here ?
Here is quick copy/paste example (used on console app) to specific page on web:
try
{
WebClient webClient = new WebClient();
string content = webClient.DownloadString("http://he.wikisource.org/wiki/%D7%A9%D7%95%D7%9C%D7%97%D7%9F_%D7%A2%D7%A8%D7%95%D7%9A_%D7%90%D7%95%D7%A8%D7%97_%D7%97%D7%99%D7%99%D7%9D_%D7%90_%D7%90");
}
catch (Exception ex)
{
throw;
}
I've just tried it with Fiddler running to see the response and it returns the following notice with the status code.
Scripts should use an informative User-Agent string with contact information, or they may be IP-blocked without notice.
This works.
WebClient webClient = new WebClient();
webClient.Headers.Add("user-agent", "Only a test!");
string content = webClient.DownloadString("http://he.wikisource.org/wiki/%D7%A9%D7%95%D7%9C%D7%97%D7%9F_%D7%A2%D7%A8%D7%95%D7%9A_%D7%90%D7%95%D7%A8%D7%97_%D7%97%D7%99%D7%99%D7%9D_%D7%90_%D7%90");