407 Authentication required - no challenge sent

Question

Update:
If you've just arrived at this question, the general gist is that I'm trying to make a HttpWebRequest via a proxy, and I'm getting a 407 from our strange proxy server. IE,Firefox,Chrome all manage to negotiate the proxy sucessfully, as do Adobe Air applications. It may be important that the Google Chrome web installer actually fails and we have to use an offline installer.

Thanks to Ian's link I've got it getting through to the next stage. It is now sending a token back to the proxy, however the 3rd stage isn't getting through, so the request with the username/password hash isn't being sent by .NET and consequently no HTML is returned.

I am using:

• IE6 user-agent
• Windows 7
• Scansafe proxy
• .NET 3.5

Here's the latest code that equates to the logs below:

HttpWebRequest request = HttpWebRequest.Create("http://www.yahoo.com") as HttpWebRequest;
IWebProxy proxy = request.Proxy;
// Print the Proxy Url to the console.
if (proxy != null)
{
    // Use the default credentials of the logged on user.
    proxy.Credentials = CredentialCache.DefaultCredentials;
}
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)";
request.Accept = "*/*";

HttpWebResponse response = request.GetResponse() as HttpWebResponse;
Stream stream = response.GetResponseStream();

The exception

WebException (407) Authentication Required.

The proxy being used

The proxy client is a Scansafe hardware device in our server room, which (once authenticated with NTLM) then directs your HTTP traffic to its servers to filter the traffic.

System.Net tracing output

• Here's the trace output

IE sucessfully negotiating the proxy

• Fidder output

The solution

I haven't really found a solution but thanks to Feroze and Eric I have found a workaround and discovered that the actual proxy (and not its configuration) is the main issue. It may be an obscure issue with 3 variables: .NET HttpWebRequest's implementation, Windows 7 and of course the Scansafe hardware client that sits in our rack; but without an MSDN support request I won't find out.

Answer 1

If you want to set credentials for the proxy, shouldn't you set credentials on the request.Proxy object rather than the request object?

http://msdn.microsoft.com/en-us/library/system.net.webproxy.credentials.aspx

Also, keep in mind that you need to be making a HTTP/1.1 request (or technically, any request with Keep-Alive) to successfully use NTLM/Negotiate authentication.

(Fiddler's "Auth" inspector will decompose the NTLM authentication blobs for you, if you haven't taken a look at that yet.)