ASP.NET Custom 404 Returning 200 OK Instead of 404 Not Found

Question

After trying to setup my site for Google Webmaster Tools I found that my Custom ASP.NET 404 page was not returning the 404 status code. It displayed the correct custom page and told the browser that everything is OK. This is consider a soft 404 or false 404. Google doesn't like this. So I found many articles on the issue but the solution I want didn't seem to work.

The solution I want to work is adding the following two lines to the code behind Page_Load method of the custom 404 page.

Response.Status = "404 Not Found";
Response.StatusCode = 404;

This doesn't work. The page still returns 200 OK. I found however that if I hard code the following code into the design code it will work properly.

<asp:Content ID="ContentMain" ContentPlaceHolderID="ContentPlaceHolderMaster" runat="server">

<%
    Response.Status = "404 Not Found";
    Response.StatusCode = 404;
%>

 ... Much more code ...

</asp:content>

The page is using a master page. And I am configuring custom error pages in my web.config. I would really rather use the code behind option but I can't seem to make it work without putting a the hack inline code in the design / layout.

Answer 1

Solution:

The problem, it turned out, was the use of the master page. I got it to work by setting the status code later in the pages lifecycle, obviously the rendering of the master page was resetting it, so I overrode the render method and set it after the render was complete.

protected override void Render(HtmlTextWriter writer)
{
    base.Render(writer);
    Response.StatusCode = 404;
}

More work could be done to find out exactly when the master page is setting the status, but I'll leave that to you.

---

Original Post:

I was able to get a test web app to work fine, well it at least displayed the custom error page and returned a 404 status code. I can't tell you what is wrong with your app, but I can tell you what I did:

1) Edited the web.config for custom errors:

<customErrors mode="On">
  <error statusCode="404" redirect="404.aspx"/>
</customErrors>

2) Added a 404.aspx page and set the status code to 404.

public partial class _04 : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        Response.StatusCode = 404;
    }
}

Thats about it, if I go to any page extension that is processed by Asp.Net and does not exist, my fiddler log clearly shows a 404, here is the header:

HTTP/1.1 404 Not Found
Server: Microsoft-IIS/5.1
Date: Sun, 07 Dec 2008 06:04:13 GMT
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 533

Now if I go to a page that is not processed by Asp.Net, like a htm file, the custom page does not show and the 404 that is configured by IIS is displayed.

Here is a post that goes into some more details that may be of use to you and your problem, my test does do a redirect to the new page so the url of the requested file is pretty much lost (except its in the query string).

Google 404 and .NET Custom Error Pages

Header Spy Response:

HTTP/1.1 404 Not Found
Date: Sun, 07 Dec 2008 06:21:20 GMT