Integration vs acceptance test ... what is Cucumber / Steak?
For integration tests of my Rails web app I use Steak (something like Cucumber). The specs of Steak are in a folder named spec/acceptance. Are Steak/Cucumber now for integration or acceptance testing? I always thought that this is something different.
Answer
First, a note on the terminology: the term integration test is a bit vague in the TDD community. Depending whether you come from Java or Rails (with Test::Unit), you might understand different things by it. In Rails (with Test::Unit) integration tests are the tests that test your full stack, while functional tests would be the ones testing your controller. Most people in the Java community (at least by my observation) would think it is the other way around. I personally prefer to call the end-to-end tests acceptance tests, while tests that hit several layers of the system (but not everything) -- integration tests. All in all, that is pretty dependent on the culture your are in.
As for Cucumber and Steak -- both are frameworks that allow a development style known as Behavior-Driven Development (or BDD for short). The point is that you have two levels of tests:
- End-to-end tests, which test your through the full stack -- they simulate a browser, go through your controllers and hit the database. Cucumber and Steak fit this niche.
- Unit tests, which test a small bit of functionality in isolation (usually a single class, mocking its collaborators). This is where RSpec fits.
In BDD, you start with a failing end-to-end test (lovingly know as the "upper gear"), and then you start implementing functionality test-first with RSpec (the "lower gear"), until you get the end-to-end test passing. This way the end-to-end test is driving your unit tests, which in turn are driving your implementation. The main benefit is avoiding scope creep -- you don't end up implementing user-visible functionality that you don't need (since you don't write an end-to-end test for it).
If you want more information on this, I've heard that the Behavior Driven Development Wikipedia article is surprisingly good. Also, the RSpec book.
So, both Cucumber and Steak are frameworks that allow you to write tests in the "upper gear". The difference is in the style -- Cucumber has you writing your tests in natural language. This has several benefits.
- Tests are readable by the business people -- while you cannot expect non-programmers to write them, they do a great job in communicating what you intend to do. You can write the feature (the Cucumber test first) and show it to the customer to get some feedback on whether this is what they actually want. I've found this very useful.
- Cucumber features communicate intent better -- since you get to use the full power of the English language (or any, really), you can communicate why this feature is relevant and how the users accomplish their goal on a level that Ruby won't allow you to.
- Cucumber helps discovering the ubiquitous language -- the domain includes a lot of terms that fly around in the conversations with the customers. Cucumber allows you to discover and capture them before you start implementing the feature. And it's all test-driven.
- Cucumber features are somewhat higher-level, which makes the features (but not the step definitions) more independent of the interface. This way if the interface needs changing, you won't have to rework the features.
The downsides include that it is a bit tricky to learn how to apply it nicely and that you have to write a bit more (both features and step definitions). I've found that the second is not really a problem if you have been doing it for a while, since you get a body of reusable steps that allow you to write the next features faster.
Steak, on the other hand is simpler and it's Ruby. You loose all the benefits of using English, but you can write less and it will execute faster (somewhat).
In the bottom line, you use both to write the end-to-end tests that drive development.