HttpWebRequest is extremely slow!

c# performance http httpwebrequest
Earlz picture Earlz · Mar 25, 2010 · Viewed 60.1k times · Source

I am using an open source library to connect to my webserver. I was concerned that the webserver was going extremely slow and then I tried doing a simple test in Ruby and I got these results

Ruby program: 2.11seconds for 10 HTTP GETs

Ruby program: 18.13seconds for 100 HTTP GETs

C# library: 20.81seconds for 10 HTTP GETs

C# library: 36847.46seconds for 100 HTTP GETs

I have profiled and found the problem to be this function:

private HttpWebResponse GetRawResponse(HttpWebRequest request) {
  HttpWebResponse raw = null;
  try {
    raw = (HttpWebResponse)request.GetResponse(); //This line!
  }
  catch (WebException ex) {
    if (ex.Response is HttpWebResponse) {
      raw = ex.Response as HttpWebResponse;
    }
  }
  return raw;
}

The marked line is takes over 1 second to complete by itself while the ruby program making 1 request takes .3 seconds. I am also doing all of these tests on 127.0.0.1, so network bandwidth is not an issue.

What could be causing this huge slow down?

UPDATE

Check out the changed benchmark results. I actually tested with 10 GETs and not 100, I updated the results.

Answer

James Roland picture James Roland · Aug 30, 2010

What I have found to be the main culprit with slow web requests is the proxy property. If you set this property to null before you call the GetResponse method the query will skip the proxy autodetect step:

request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}

The proxy autodetect was taking up to 7 seconds to query before returning the response. It is a little annoying that this property is set on by default for the HttpWebRequest object.

Related questions

Best way to reverse a string

I've just had to write a string reverse function in C# 2.0 (i.e. LINQ not available) and came up with this: public string Reverse(string text) { char[] cArray = text.ToCharArray(); string reverse = String.Empty; for (int i = cArray.Length - 1; …

Read More
How to create a new object instance from a Type

One may not always know the Type of an object at compile-time, but may need to create an instance of the Type. How do you get a new object instance from a Type?

Read More
Make first letter of a string upper case (with maximum performance)

I have a DetailsView with a TextBox and I want the input data be saved always with the FIRST LETTER IN CAPITAL. Example: "red" --> "Red" "red house" --> " Red house" How can I achieve this maximizing performance? …

Read More