What is the purpose of XCTestCase's setUp method?

Question

Per the comment within the default template for XCTestCase regarding setUp :

Put setup code here; it will be run once, before the first test case.

However, in XCTestCase.h, the comment above setUp states differently:

Setup method called before the invocation of each test method in the class.

To confirm the actual behavior, I put an NSLog withinsetUp to count how many times it was called:

static int count = 0;

- (void)setUp
{
    [super setUp];
    count++;

    NSLog(@"Call Count = %d", count);
}

This resulted in the setUp method being called before every test method (confirming the comment on XCTestCase.h).

I wanted to use the setUp method to create test/mock objects once (e.g. to setup a Core Data test stack). Creating these over and over again would be processor intensive and potentially very slow.

So,

1) What is setUp actually intended to be used for? Surely developers aren't creating objects in it over and over?

2) How can I create these objects only once within an XCTestCase?

Answer 1

There are a couple of points for discussion here: the behaviour of the setUp methods, and general best testing practices.

There are actually two setUp methods:

+ (void)setUp;
- (void)setUp;

The class method (+ (void)setUp) is only run once during the entire test run.

The instance method (- (void)setUp) is the one in the default template; it's run before every single test. Hopefully, in a hypothetical future version of Xcode, this comment will have been changed to // Put setup code here. This method is called before the invocation of each test method in the class. WINK WINK

So through these two methods, both of the behaviours you described are possible.

Regarding your comment:

"Surely developers aren't creating objects in it over and over?"

My answer would be "Yes, they usually are". A popular acronym for 'good' unit tests is FIRST:

• Fast
• Isolated
• Repeatable
• Self-Verifying
• Timely

Isolated is key to this discussion: your tests shouldn't rely on any previous state left behind by other tests. Ideally, you should tear down and recreate your in-memory Core Data stack for every test, so you know that you're starting from a clean slate. A good example is in this post by Graham Lee. You want to use an in-memory stack because a) you can easily throw it away, and b) it should be very fast because it's just in-memory and not hitting your disk.

If you find that your tests are running slowly as a result of this (don't optimize prematurely), then I think a reasonable next step would be to create the stack in your + (void)setUp method, but create a brand new context every time in your - (void)setUp method.