Is it possible to use Selenium WebDriver to drive PhantomJS?

selenium web-scraping webdriver phantomjs
Joseph picture Joseph · Jun 27, 2012 · Viewed 34.1k times · Source

I’m going through the documentation for the Selenium WebDriver, and it can drive Chrome for example. I was thinking, wouldn't it be far more efficient to ‘drive’ PhantomJS?

Is there a way to use Selenium with PhantomJS?

My intended use would be web scraping: The sites I scrape are loaded with AJAX and lots of lovely JavaScript, and I’m thinking this setup could be a good replacement for the Scrapy Python framework that I’m currently working with.

Answer

Ariya Hidayat picture Ariya Hidayat · Jun 27, 2012

PhantomJS now includes the GhostDriver project.

You are also suggested to use PhantomJS directly or with a convenience library such as CasperJS. CasperJS is specifically designed to make it easy to do sequential operations to web pages, perfect for many automation tasks.

Disclaimer: I am the author of PhantomJS.

Edit: As noted in Nick's answer, GhostDriver is now included in PhantomJS.

Related questions

Click the javascript popup through webdriver

I am scraping a webpage using Selenium webdriver in Python The webpage I am working on, has a form. I am able to fill the form and then I click on the Submit button. It generates an popup window( Javascript …

Read More
selenium with scrapy for dynamic page

I'm trying to scrape product information from a webpage, using scrapy. My to-be-scraped webpage looks like this: starts with a product_list page with 10 products a click on "next" button loads the next 10 products (url doesn't change between the two …

Read More
How can I download a file on a click event using selenium?

I am working on python and selenium. I want to download file from clicking event using selenium. I wrote following code. from selenium import webdriver from selenium.common.exceptions import NoSuchElementException from selenium.webdriver.common.keys import Keys browser = webdriver.…

Read More