For some time I have been investigating Selenium RC in order to do functional testing of my web application. I have now found a test strategy that is so effective, that I do not want to move away from Selenium RC (after spending weeks trying to figure out a good way to validate ASP.NET validation controls).
But now that my Selenium RC adventure is moving from a POC to be something that I actually use, I'm running into a problem. It is insanely slow. Executing a single test that loads a page, fills in some fields, and clicks a button takes in the magnitude of seconds to execute. When it is executing, I can easily see each individual field being filled out one at a time. Using Selenium IDE in Firefox is not that slow.
I found this page, that clearly specifies that Selenium RC is slow http://selenium-grid.seleniumhq.org/how_it_works.html
But why is that? Is it because the browser is polling the selenium server? If so, can this polling interval not be modified? Or is there another reason. I am not accustomed to a remote call taking a humanly noticable amount of time to execute.
It is horrible that executing a few tests should take so long. I can execute my entire presentation (MVP), business, and database layer test suite (500+ tests) way quicker than it takes to run 10 tests for a single web page.
Answer
Functional/Integration tests will take longer to run especially since they are running in a Browser. This means that it they have to load up all 3 layers of your MVC and then execute and the same when it is doing anything on the page. So every action has the potential to go down to the database. This is inherently a long running tasks compared to unit tests.
The tests start by doing an open on that page which then waits for everything to load. So if this is taking a long time then it could be taking a long time for your user if they were to access the page. E.g. Lots of images, unminified JavaScript/CSS, poor expiries on downloads.
What that page from Selenium is saying that the server is a bottleneck because it implies that you are running the tests synchronisely and if you moved to Selenium Grid it can run them in parallel to make the test suite complete faster. It is not suggesting that the selenium server is polling to see what it should do but instead the Selenium Servers poll the Grid hub to see if it is still alive and to show they are still alive.
The other reason the tests are running slow is Selenium's base language is JavaScript which interacts with the DOM. The DOM can slow things down a lot especially if your tests are using XPath as locators. XPath + JavaScript + IE + Selenium == Painful and there is nothing that we Selenium Developers can do more to fine tune it. Well there is and that is going to be Selenium 2 which is in alpha and can be downloaded from http://selenium.googlecode.com/ . I have been working on the .NET implementation am seeing huge speed improvements at the moment. I have blogged about it because the changes astounded me. I was seeing upto 8 tests running in the same time it used to take Selenium 1 to run 1 test