Testing Storm Bolts and Spouts

unit-testing junit apache-storm
Jack picture Jack · May 14, 2013 · Viewed 16.8k times · Source

This is a general question regarding Unit Testing Bolts and Spouts in a Storm Topology written in Java.

What is the recommended practice and guideline for unit-testing (JUnit?) Bolts and Spouts?

For instance, I could write a JUnit test for a Bolt, but without fully understanding the framework (like the lifecycle of a Bolt) and the Serialization implications, easily make the mistake of Constructor-based creation of non-serializable member variables. In JUnit, this test would pass, but in a topology, it wouldn't work. I fully imagine there are many test points one needs to consider (such as this example with Serialization & lifecycle).

Therefore, is it recommended that if you use JUnit based unit tests, you run a small mock topology (LocalMode?) and test the implied contract for the Bolt (or Spout) under that Topology? Or, is it OK to use JUnit, but the implication being that we have to simulate the lifecycle of a Bolt (creating it, calling prepare(), mocking a Config, etc) carefully? In this case, what are some general test points for the class under test (Bolt/Spout) to consider?

What have other developers done, with respect to creating proper unit tests?

I noticed there is a Topology testing API (See: https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java). Is it better to use some of that API, and stand up "Test Topologies" for each individual Bolt & Spout (and verifying the implicit contract that the Bolt has to provide for, eg - it's Declared outputs)?

Thanks

Answer

asmaier picture asmaier · Oct 9, 2013

Since version 0.8.1 Storm's unit testing facilities have been exposed via Java:

For an example how to use this API have a look here:

Related questions

Mockito : how to verify method was called on an object created within a method?

I am new to Mockito. Given the class below, how can I use Mockito to verify that someMethod was invoked exactly once after foo was invoked? public class Foo { public void foo(){ Bar bar = new Bar(); bar.someMethod(); } } I would …

Read More
How to run test methods in specific order in JUnit4?

I want to execute test methods which are annotated by @Test in specific order. For example: public class MyTest { @Test public void test1(){} @Test public void test2(){} } I want to ensure to run test1() before test2() each time I run …

Read More
How to run JUnit test cases from the command line

I would like to run JUnit test cases from the command line. How can I do this?

Read More