MemberData tests show up as one test instead of many

Question

When you use [Theory] together with [InlineData] it will create a test for each item of inline data that is provided. However, if you use [MemberData] it will just show up as one test.

Is there a way to make [MemberData] tests show up as multiple tests?

Answer 1

I spent a lot of time trying to figure this one out in my project. This related Github discussion from @NPadrutt himself helped a lot, but it was still confusing.

The tl;dr is this: [MemberInfo] will report a single group test unless the provided objects for each test can be completely serialized and deserialized by implementing IXunitSerializable.

---

Background

My own test setup was something like:

public static IEnumerable<object[]> GetClients()
{
    yield return new object[] { new Impl.Client("clientType1") };
    yield return new object[] { new Impl.Client("clientType2") };
}

[Theory]
[MemberData(nameof(GetClients))]
public void ClientTheory(Impl.Client testClient)
{
    // ... test here
}

The test ran twice, once for each object from [MemberData], as expected. As @NPadrutt experienced, only one item showed up in the Test Explorer, instead of two. This is because the provided object Impl.Client was not serializable by either interface xUnit supports (more on this later).

In my case, I didn't want to bleed test concerns into my main code. I thought I could write a thin proxy around my real class that would fool the xUnit runner into thinking it could serialize it, but after fighting with it for longer than I'd care to admit, I realized the part I wasn't understanding was:

The objects aren't just serialized during discovery to count permutations; each object is also deserialized at test run time as the test starts.

So any object you provide with [MemberData] must support a full round-trip (de-)serialization. This seems obvious to me now, but I couldn't find any documentation on it while I was trying to figure it out.

---

Solution

• Make sure every object (and any non-primitive it may contain) can be fully serialized and deserialized. Implementing xUnit's IXunitSerializable tells xUnit that it's a serializable object.

• If, as in my case, you don't want to add attributes to the main code, one solution is to make a thin serializable builder class for testing that can represent everything needed to recreate the actual class. Here's the above code, after I got it to work:

TestClientBuilder

public class TestClientBuilder : IXunitSerializable
{
    private string type;

    // required for deserializer
    public TestClientBuilder()
    {
    }

    public TestClientBuilder(string type)
    {
        this.type = type;
    }

    public Impl.Client Build()
    {
        return new Impl.Client(type);
    }

    public void Deserialize(IXunitSerializationInfo info)
    {
        type = info.GetValue<string>("type");
    }

    public void Serialize(IXunitSerializationInfo info)
    {
        info.AddValue("type", type, typeof(string));
    }

    public override string ToString()
    {
        return $"Type = {type}";
    }
}

Test

public static IEnumerable<object[]> GetClients()
{
    yield return new object[] { new TestClientBuilder("clientType1") };
    yield return new object[] { new TestClientBuilder("clientType2") };
}

[Theory]
[MemberData(nameof(GetClients))]
private void ClientTheory(TestClientBuilder clientBuilder)
{
    var client = clientBuilder.Build();
    // ... test here
}

It's mildly annoying that I don't get the target object injected anymore, but it's just one extra line of code to invoke my builder. And, my tests pass (and show up twice!), so I'm not complaining.