I'm trying to parse the frontpage of facebook with JSoup but I always get the HTML Code for mobile devices and not the version for normal browsers(In my case Firefox 5.0).
I'm setting my User Agent like this:
doc = Jsoup.connect(url)
.userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0")
.get();
Am I doing something wrong?
EDIT:
I just parsed http://whatsmyuseragent.com/ and it looks like the user Agent is working. Now its even more confusing for me why the site http://www.facebook.com/ returns a different version when using JSoup and my browser. Both are using the same useragent....
I noticed this behaviour on some other sites too now. If you could explain to me what the Issue is I would be more than happy.
Answer
You might try setting the referrer header as well:
doc = Jsoup.connect("https://www.facebook.com/")
.userAgent("Mozilla/5.0 (Windows; U; WindowsNT 5.1; en-US; rv1.8.1.6) Gecko/20070725 Firefox/2.0.0.6")
.referrer("http://www.google.com")
.get();