Is Selenium testing worthwhile on mobile devices?
We've been planning to do UI testing of our web application using Selenium. We're now having second thoughts. From what I can tell, the webdrivers for Android and iOS devices don't use the actual browser that a user would use. Instead they use a WebView, which may or may not be similar to the actual browser. I've seen various reports for Android in particular that seem to indicate that the WebView behaves differently than the built in browser and from Chrome, which is now the default on some devices.
Do you find that using Selenium to test on the actual hardware device is worthwhile? Is it more or less accurate than using emulators? What have you found is the most effective way to use automated testing to make you comfortable that your web application will work correctly on all of the hardware platforms that you support?
In our case, we need to support recent versions of major browsers on PCs, Macs, tablets, and phones. Selenium is working well for the PC testing, but I'm concerned about the tablet and phone testing. We're considering whether we should roll our own or use something like uitest.js that we can actually run on the actual device browsers.
Answer
While both the iPhone driver and Android driver that exist in the Selenium WebDriver source code repository today use a WebView control to browse the web, both projects have been largely deprecated. This deprecation is explicit in the case of the iPhone driver, and tacit in the case of the Android driver.
Those drivers have been supplanted by other projects like ios-driver, Selendroid, and Appium. These projects are based on the WebDriver project, using the same client API and communicate using the same JSON-over-HTTP-based wire protocol, but are capable of automating native and hybrid applications on mobile platforms. They can usually automate the application either on the device or in an emulator. In the case of each of the three projects I've mentioned, they are actively maintained and development is ongoing.