How to execute cucumber test cases in parallel using Grid?

Question

After doing a fair amount of research on how to run Cucumber test cases in parallel, I found the following very useful article on the subject matter:

https://www.opencredo.com/2013/07/02/running-cucumber-jvm-tests-in-parallel/

The article has some pretty good information to get you started working with a multi-threaded environment, including some code you can download from Github.

https://github.com/tristanmccarthy/Cucumber-JVM-Parallel

If I understand the article correctly the driver should be configurable to work with Grid, enabling you to run multiple test cases across multiple devices. After doing some testing with the code using a chromedriver, it does seem to work as described in the article. However, once it is configured to work with Grid the test cases no longer get executed in parallel. Instead, they are executed sequentially.

Currently, I have Grid configured to have 1 hub and 2 nodes. Each node can have a maximum of 2 sessions at any given time.

Note: Without Cucumber I'm able to successfully deploy multiple test cases across multiple devices, So I don’t think the problem is related to my grid setup.

Here is a sample of the code related to the web driver:

static {
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setJavascriptEnabled(true);
    capabilities.setBrowserName("chrome");
    capabilities.setPlatform(Platform.ANY);
    try {
        REAL_DRIVER = new RemoteWebDriver(new URL("http://xxx.xxx.xxx.xxx:4444/wd/hub"), capabilities);
    } catch (MalformedURLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    REAL_DRIVER.manage().timeouts().pageLoadTimeout(3000, TimeUnit.SECONDS);
    REAL_DRIVER.manage().window().maximize();
    Runtime.getRuntime().addShutdownHook(CLOSE_THREAD);
}

public SharedDriver() {
    super(REAL_DRIVER);
}

@Override
public void close() {
    if (Thread.currentThread() != CLOSE_THREAD) {
        throw new UnsupportedOperationException(
                "You shouldn't close this WebDriver. It's shared and will close when the JVM exits.");
    }
    super.close();
}

I suspect that if you use more than one browser type that you should be able to run the test cases on multiple devices (1 browser per device), but in my case I am on using Chrome driver. Does anyone know what might be preventing the test cases from being distributed across multiple devices or have any better understanding of how Grid works with cucumber? Please share any articles or information related to this problem.

Answer 1

Grid doesn't do anything about splitting jobs up. It took me awhile (and a lot of scotch) to finally realize all Grid does is take a job from somewhere and pass it to an available node.

You need a test runner that will split your feature/scenario tests in to different chunks to pass to Grid. Unfortunately, the Cucumber runner doesn't do that. There are several different ways to split those tests into separate jobs to pass to the Grid.

That OpenCredo blog points to a newer post that uses Maven. Make sure to check that out!

Someone mentioned TestNG. I've not used it, so I can't comment on it.

You can split up your features/scenarios yourself and pass jobs separately to your Grid by running different test passes--that's cumbersome for long-term maintainability, but it's a quick start.

We wrote a small runner that scans scenarios and dynamically passes them to the Grid. Can't share code because it's at work and I'm in my hotel...

One thing to keep in mind: you'll have to manage dependencies and concurrency issues. Hopefully you're structuring tests so there's no dependencies between them. Concurrency's a different matter. We've got a bit of code that hands unlocked resources to the tests (think users, data sets, etc.)

Good luck!