XCTAssertEqual error: ("3") is not equal to ("3")

Question

NSMutableArray *arr = [NSMutableArray array];
[arr addObject:@"1"];
[arr addObject:@"2"];
[arr addObject:@"3"];

// This statement is fine.
XCTAssertTrue(arr.count == 3, @"Wrong array size.");

// This assertion fails with an error: ((arr.count) equal to (3)) failed: ("3") is not equal to ("3")
XCTAssertEqual(arr.count, 3, @"Wrong array size.");

What don't I understand about XCTAssertEqual? Why does the last assertion fail?

Answer 1

I have also had quite a bit of trouble with Xcode 5's tests. It still seems quite buggy with some odd behaviour - however I have found the definitive reason your particular XCTAssertEqual isn't working.

If we have a look at the test code we see it actually does the following (taken directly from XCTestsAssertionsImpl.h - it may be easier to view there):

#define _XCTPrimitiveAssertEqual(a1, a2, format...) \
({ \
    @try { \
        __typeof__(a1) a1value = (a1); \
        __typeof__(a2) a2value = (a2); \
        NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))]; \
        NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))]; \
        float aNaN = NAN; \
        NSValue *aNaNencoded = [NSValue value:&aNaN withObjCType:@encode(__typeof__(aNaN))]; \
        if ([a1encoded isEqualToValue:aNaNencoded] || [a2encoded isEqualToValue:aNaNencoded] || ![a1encoded isEqualToValue:a2encoded]) { \
                _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 0, @#a1, @#a2, _XCTDescriptionForValue(a1encoded), _XCTDescriptionForValue(a2encoded)),format); \
        } \
    } \
    @catch (id exception) { \
        _XCTRegisterFailure(_XCTFailureDescription(_XCTAssertion_Equal, 1, @#a1, @#a2, [exception reason]),format); \
    }\
})

Here's the problem:

What the test is actually doing is encoding the values into an NSValue and then comparing them. "Okay," you say, "but what's the problem with that?" I didn't think there was one either until I made my own test case for it. The problem is that NSValue's -isEqualToValue must also compare the NSValue's encoding type as well as its actual value. Both must be equal for the method to return YES.

In your case, arr.count is an NSUInteger which is a typedef of unsigned int. The compile-time constant 3 presumably degenerates into a signed int at runtime. Thus when the two are put into an NSValue object, their encoding types are not equal and thus the two CANNOT be equal according to -[NSValue isEqualToValue].

You can prove this with a custom example. The following code explicitly does exactly what XCTAssertEqual does:

// Note explicit types
unsigned int a1 = 3;
signed int a2 = 3;

__typeof__(a1) a1value = (a1);
__typeof__(a2) a2value = (a2);

NSValue *a1encoded = [NSValue value:&a1value withObjCType:@encode(__typeof__(a1))];
NSValue *a2encoded = [NSValue value:&a2value withObjCType:@encode(__typeof__(a2))];

if (![a1encoded isEqualToValue:a2encoded]) {
    NSLog(@"3 != 3 :(");
}

"3 != 3 :(" will appear in the log every time.

I hasten to add here that this is, in fact, expected behaviour. NSValue is supposed to check its type encoding when doing comparisons. Unfortunately it's just not what we were expecting when testing two ('equal') integers.

XCTAssertTrue, incidentally, has much more straightforward logic, and behaves generally as expected (again, see the actual source for how it determines whether the assertion fails).