How is white box testing done on java applications? Does it mean that I have to test class by class?
From my research I know that:
White Box Testing is a software testing method in which the internal structure/design/implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White box testing is testing beyond the user interface and into the nitty-gritty of a system.
What should the white box testing look like?
Answer
Summary: Modern white-box testing uses sophisticated automated tools to help developers design good tests and measure code coverage in meaningful ways. Widely used (if rudimentary) examples include Cobertura, EMMA, EclEmma and JaCoCo.
How is white box testing done on java applications?
Traditionally, black-box testing meant that another team (not the developer) was responsible for testing the software, and they did not look at the source code while doing so.
Even with the move to unit testing, a common approach to designing tests is to use a more-or-less black-box input domain characterization, that is, just test the boundary conditions and a few arbitrary "normal" input values.
Clearly, however, this approach may fail to exercise some branches of the function. A good developer will open up the code and say "hey, I need to add an input to exercise that branch."
Modern white-box testing takes that idea a step further by automating the process, producing output like this to show which parts of a method have not yet been exercised by, say, JUnit (from the NetBeans plugin TikiOne):
What should the white box testing look like?
The code coverage tools mentioned in the summary attempt to automate the process of making sure you exercise every line or branch of code. When integrated into a continuous integration environment like Jenkins, it allows your team to keep a constant eye on not only whether your tests are passing, but whether you are meeting your coverage goals (from the NetBeans plugin TikiOne):
In software engineering research, white-box testing can get far more sophisticated than counting lines. A wide variety of tools and mathematical techniques have been developed to help you make sure that the inputs you give to your tests properly cover all the possible behaviors of your code.
For instance, warning systems for an airplane auto-pilot system that uses complex Boolean predicates might use white-box testing criteria rooted in formal logic to ensure that we cover all logically possible paths through the controller.
Analyzing line and branch coverage, the most common criteria, is a special case of using graph representations of software, such as a control flow graph or a data flow diagram. A good tool will read your code to generate the graph, and then you can choose from various graph coverage criteria to automatically recommend a set of paths through the method's control logic (branches) that you need to exercise in order to fulfill your testing goals.
Lines and branches are the simplest and most-used (control) graph coverage criteria (corresponding to covering nodes and edges) -- more thorough (and difficult to satisfy) metrics like edge-pairs and prime paths are available in some tools. These ask not just "did I go down every choice at each branch at least once," but "did I go through every branch via every way I was able to get there."
It's still up to the programmer to find a set of inputs to exercise these paths, and to design code small enough and simple enough that the number of required tests to get good coverage with the more thorough criteria doesn't explode exponentially. But now you have formal, quantitative ways to A) tell you what needs tested, and B) tell you how well you have tested your project.
Does it mean that I have to test class by class?
White-box testing is a method for designing tests. It can apply at all testing levels, including unit tests, integration tests, and system tests.
For more information on test coverage criteria, see
- Beizer, Boris. Software testing techniques. Dreamtech Press, 2003.
or see the more up-to-date
- Ammann, Paul, and Jeff Offutt. Introduction to software testing. Cambridge University Press, 2008.