XCTestExpectation: how to avoid calling the fulfill method after the wait context has ended?

Question

I’m using the new asynchronous testing capabilities of Xcode 6. Everything works fine when the asynchronous task ends before the timeout. But if the task takes longer than the timeout, things get more complicated.

Here is how I’m doing my tests:

@interface AsyncTestCase : XCTestCase @end

@implementation AsyncTestCase

// The asynchronous task would obviously be more complex in a real world scenario.
- (void) startAsynchronousTaskWithDuration:(NSTimeInterval)duration completionHandler:(void (^)(id result, NSError *error))completionHandler
{
    dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(duration * NSEC_PER_SEC)), dispatch_get_main_queue(), ^{
        completionHandler([NSObject new], nil);
    });
}

- (void) test1TaskLongerThanTimeout
{
    XCTestExpectation *expectation = [self expectationWithDescription:@"Test 1: task longer than timeout"];
    [self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
        XCTAssertNotNil(result);
        XCTAssertNil(error);
        [expectation fulfill];
    }];
    [self waitForExpectationsWithTimeout:2 handler:nil];
}

- (void) test2TaskShorterThanTimeout
{
    XCTestExpectation *expectation = [self expectationWithDescription:@"Test 2: task shorter than timeout"];
    [self startAsynchronousTaskWithDuration:5 completionHandler:^(id result, NSError *error) {
        XCTAssertNotNil(result);
        XCTAssertNil(error);
        [expectation fulfill];
    }];
    [self waitForExpectationsWithTimeout:10 handler:nil];
}

@end

Unfortunately, calling the fulfill method after the timeout has expired crashes the test suite with this error:

API violation - called -[XCTestExpectation fulfill] after the wait context has ended.

*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'API violation - called -[XCTestExpectation fulfill] after the wait context has ended.'
*** First throw call stack:
(
  0   CoreFoundation   0x000000010c3a6f35 __exceptionPreprocess + 165
  1   libobjc.A.dylib  0x000000010a760bb7 objc_exception_throw + 45
  2   CoreFoundation   0x000000010c3a6d9a +[NSException raise:format:arguments:] + 106
  3   Foundation       0x000000010a37d5df -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 195
  4   XCTest           0x0000000115c48ee1 -[XCTestExpectation fulfill] + 264
  ...
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Of course I can check if the test is finished before calling the fulfill method like this:

- (void) test1TaskLongerThanTimeout
{
    XCTestExpectation *expectation = [self expectationWithDescription:@"Test 1: task longer than timeout"];

    __block BOOL testIsFinished = NO;
    [self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
        if (testIsFinished) {
            return;
        }
        XCTAssertNotNil(result);
        XCTAssertNil(error);
        [expectation fulfill];
    }];

    [self waitForExpectationsWithTimeout:2 handler:^(NSError *error) {
        testIsFinished = YES;
    }];
}

But this seems overly complicated and makes the test much harder to read. Am I missing something? Is there a simpler way to solve this problem?

Answer 1

Yes, there is a much simpler way to avoid this API violation issue: just declare your expectation variable as __weak. Although not clearly documented, the expectation will be released when the timeout expires. So if the task takes longer than the timeout, the expectation variable will be nil when the task completion handler is called. Thus the fulfill method will be called on nil, doing nothing.

- (void) test1TaskLongerThanTimeout
{
    __weak XCTestExpectation *expectation = [self expectationWithDescription:@"Test 1: task longer than timeout"];
    [self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
        XCTAssertNotNil(result);
        XCTAssertNil(error);
        [expectation fulfill];
    }];
    [self waitForExpectationsWithTimeout:2 handler:nil];
}