Alternative to HtmlUnit

screen-scraping web-crawler htmlunit headless-browser
Nayn picture Nayn · Nov 23, 2010 · Viewed 12.8k times · Source

I have been researching about the headless browsers available till to date and found HtmlUnit being used pretty extensively. Do we have any alternative to HtmlUnit with possible advantage compared to HtmlUnit?

Thanks Nayn

Answer

Ahmed Ashour picture Ahmed Ashour · Nov 23, 2010

As far as I know, HtmlUnit` is the most powerful headless browser.

What are you issues with it?

Related questions

Scrapy Python Set up User Agent

I tried to override the user-agent of my crawlspider by adding an extra line to the project configuration file. Here is the code: [settings] default = myproject.settings USER_AGENT = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36" [deploy] #…

Read More
How to use Goutte

Issue: Cannot fully understand the Goutte web scraper. Request: Can someone please help me understand or provide code to help me better understand how to use Goutte the web scraper? I have read over the README.md. I am looking …

Read More
Crawling and Scraping iTunes App Store

I noticed that iTunes preview allows you to crawl and scrape pages via the http:// protocol. However, many of the links are trying to be opened in iTunes rather than the browser. For example, when you go to the iBooks …

Read More