Sikuli, selenium testing on Jenkins: Allow the browser to be launched in the foreground, as it is when i run it from development machine?

selenium jenkins continuous-integration webdriver sikuli
user2387855 picture user2387855 · Jun 24, 2013 · Viewed 7.2k times · Source

The issue is that Sikuli's image recognition capabilities only work when the target of the Sikuli tests is in the foreground and it has full access to the mouse. In its current configuration, Jenkins projects are never visible from the desktop of the windows slave node.

The way my current automated testing suite is setup right now, about 30 tests are written using only selenium. In the final two tests, selenium starts a webdriver (currently chromedriver but can be firefox or IE) and navigates to the right pages and Sikuli does its thing from there. Both use the java language bindings.

Currently Sikuli appears to be installed correctly, but when it is asked to click through a list of screenRegions supposedly populated by matching the desktop with a target image, it immediately throws a NPE.

How do I change my Jenkins setup on my windows 7 slave to allow the browser to be launched in the foreground, as it is when i run it from my windows 7 development machine?

Answer

Nathaniel Waisbrot picture Nathaniel Waisbrot · Jun 26, 2013

So you've got a development machine where Sikuli works, and a test machine where Jenkins runs Sikuli and it doesn't work.

I had this same setup (with the Sikuli tester/Jenkins slave running Windows 7). What worked for me was to have the test machine run a VNC server, log in as a the tester account over VNC, and launch the Jenkins slave as a regular application and not a service. I used UltraVNC server.

As far as I could tell, the problem stems from Windows' security measures, which are designed to prevent a remote user from controlling your machine without your presence.

If you run Jenkins as a service, it doesn't get a real desktop allocated to it. The script will run, but (as you're seeing) Sikuli won't actually be able to find anything because there's nothing for it to look at. (Selenium is inspecting the page content programmatically, so it doesn't mind that the page is not actually displayed on any screen.)

You can also just run Jenkins and leave yourself logged in, and skip the VNC server. I wanted to run the machine without a monitor or keyboard and still have control, though. If that's your goal as well, you can't use Remote Desktop, because it allocates a new desktop on connection and then destroys it when you disconnect. (So Jenkins will work when you're watching and fail when you leave for the night. Very frustrating!)

As a side benefit, if you do it this way you can connect over VNC and watch Jenkins as it's running the test.

Related questions

Setting up Continuous Integration of Protractor using Jenkins

I am writing automation test scripts using Protractor and now I need to set up the CI for this using Jenkins. Tasks it needs to perform are: Starting the selenium standalon server. Starting the test using conf.js file. Stopping …

Read More
Webdriver Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms

I have box where I run tests. It seems like Jenkins would ssh in and execute commands described in the specific job that's running. Here I am trying to run my Selenium Webdriver tests, but it tells me that I …

Read More
Jenkins : Selenium GUI tests are not visible on Windows

When I run my selenium test (mvn test) from jenkins (windows) I see only the console output. I don't see the real browsers getting opened . How can I configure jenkins so that I can see the browsers running the test?

Read More