Reason for browser not showing X-Forwarded-For headers

Question

Note: Please do read the full question

I'm trying to understand as to why the browsers doesn't show me any X-Forwarded-For header every time a request a page

BTW here are my Request Headers look like

Request URL:http://localhost:3000/users/sign_in
Request Method:GET
Status Code:304 Not Modified

Request Headers:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-GB,en-US;q=0.8,en;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Cookie:undefined=0; poasterapp=s%3A4faaa6b1723e7c6fbd949083532c52598652547b.sNX%2BKOEed2TEQkQN7I7K5lgpoHMRpwerKFvUegMnTVI; _minerva_session=BAh7CUkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiCmZsYXNoBjsARm86JUFjdGlvbkRpc3BhdGNoOjpGbGFzaDo6Rmxhc2hIYXNoCToKQHVzZWRvOghTZXQGOgpAaGFzaHsGOgphbGVydFQ6DEBjbG9zZWRGOg1AZmxhc2hlc3sGOwpJIgAGOwBUOglAbm93MEkiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--6bd89ce9d29e9bdcf56573f9a153dc663a8fe755
Host:localhost:3000
If-None-Match:"785d34e3998360353567fc710af123fb"
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.102 Safari/537.36

Response Headers(Not need but still )

Cache-Control:max-age=0, private, must-revalidate
Connection:close
ETag:"785d34e3998360353567fc710af123fb"
Server:thin 1.5.0 codename Knife
Set-Cookie:_minerva_session=BAh7CEkiD3Nlc3Npb25faWQGOgZFRkkiJWEyM2Q0ZTViMWEyODBiYmFmODEwZTJhZmUwNWU5ODk5BjsAVEkiE3VzZXJfcmV0dXJuX3RvBjsARiIGL0kiEF9jc3JmX3Rva2VuBjsARkkiMUN0Uk56SXU0dUdIdzgwcFZJM3R0L2N4dlovRllTSGRrQ2o1R0VVanhIaVk9BjsARg%3D%3D--dfb3ce9f5c97463cfcd0229a133654e6cc606d98; path=/; HttpOnly
X-Request-Id:41a6f3062dc8bc36b7b3eae71dc5075d
X-Runtime:89.238257
X-UA-Compatible:IE=Edge

Now as said, I dont see any X-Forwarded-For in request headers

Reading through the wiki pages of X-Forwarded-For make me feel that ,it is something done by caching server(which in my case I believe is my ISP provider) so am I safe to believe that the **X-Forwarded-For** headers is something that is added at the caching server side (ISP provider)

If yes their is this one bugging me about it then that is

why? is the same true (i.e X-Forwarded-For not appearing in the request-headers ) for my server running locally on my machine and I accessing them via browser like http://localhost:3000

Answer 1

X-Forwarded-For is not a standard request header as specified in RFC 2616 Section 5.3 that addresses the protocol standard request headers, which are (as specified in the RFC)

• Accept
• Accept-Charset
• Accept-Encoding
• Accept-Language
• Authorization
• Expect
• From
• Host
• If-Match
• If-Modified-Since
• If-None-Match
• If-Range
• If-Unmodified-Since
• Max-Forwards
• Proxy-Authorization
• Range
• Referer
• TE
• User-Agent

In order for your incoming request to have the custom [X-Forwarded-For] header, it must be explicitly added to that request by the calling client. The easiest explanation of why you are not seeing that header is that the client sending the request did not manually add it.

The tricky thing is that the header you are expecting to see is not a header that you should necessarily be expecting to receive unless there is a contract in place between your service and the caller that is apart from the HTTProtocol indicating that you should expect an X-Forwarded-For value to be specified in your request header. As others have already stated, the XFF header is typically set by a proxy server or a load balancer to indicate who the real requester is that is acting through their proxy.

As a service provider, if you demand that an [X-Forwarded-For] header be set in all requests, you must enforce it at a service policy level. If you do not want to service proxy accounts that do not identify who they are shielding with their proxy IP, bounce their request with a 403 Forbidden. If you are in a situation where you must service these requests but depend on this header being set, then you're going to have to come up with a custom process where you could communicate their error back.

Here is what the HTTProtocol has to say about anonymity:

Because the source of a link might be private information or might reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent. For example, a browser client could have a toggle switch for browsing openly/anonymously, which would respectively enable/disable the sending of Referer and From information.

Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
protocol.

Authors of services which use the HTTP protocol SHOULD NOT use GET
based forms for the submission of sensitive data, because this will
cause this data to be encoded in the Request-URI. Many existing
servers, proxies, and user agents will log the request URI in some
place where it might be visible to third parties. Servers can use
POST-based form submission instead ...

Elaborate user-customized accept header fields sent in every request, in particular if these include quality values, can be used by servers as relatively reliable and long-lived user identifiers. Such user identifiers would allow content providers to do click-trail tracking, and would allow collaborating content providers to match cross-server click-trails or form submissions of individual users. Note that for many users not behind a proxy, the network address of the host running the user agent will also serve as a long-lived user identifier. In environments where proxies are used to enhance privacy, user agents ought to be conservative in offering accept header configuration options to end users. As an extreme privacy measure, proxies could filter the accept headers in relayed requests. General purpose user agents which provide a high degree of header configurability SHOULD warn users about the loss of privacy which can be involved.

Personally, I would bounce the request with a 401.2 and route the requester to a challenge screen via the WWW-Authenticate response header that presents them with notification that they will not be allowed anonymous access to your site. It's kind of a bastardized way of using the WWW-Authenticate header, but it seems like you're expecting the X-Forwarded-For header to acknowledge and identify the real requester and allowing public non-anonymous access to your service. To me, that's an authentication concern.