JUnit's @TestMethodOrder annotation not working

Question

I'm having a problem with following integration test

import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.TestMethodOrder;
import org.junit.jupiter.api.MethodOrderer.OrderAnnotation;

@SpringBootTest
@ActiveProfiles("test")
@TestMethodOrder(OrderAnnotation.classs)
public class FooServiceIT {
    @Test
    @Order(1)
    void testUploadSuccess() { ... }
    @Test
    @Order(2)
    void testDownloadSuccess() { ... }
    @Test
    @Order(3)
    void testDeleteSuccess() { ... }
}

I'd expect when I run the test that the execution order would 1, 2, 3 but for some reason, the actual execution order is 2, 3, 1.

Tbh, I'm clueless why the annotation is not working. I'm using Spring Boot 2.1.3 with JUnit 5.4.

Answer 1

You need to configure correctly your IDE.

Requirements

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.4.0</version>
</dependency>

Do not use JUnit 5 that offers your IDE. If you add it as library, you will get:

No tests found for with test runner 'JUnit 5' 
==================== and this exception ===================
TestEngine with ID 'junit-vintage' failed to discover tests
java.lang.SecurityException: class "org.junit.jupiter.api.TestMethodOrder"'s signer information does not match signer information of other classes in the same package

So just include mentioned dependency only and your code will work as you expect:

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class FooServiceIT {

    @Test
    @Order(1)
    public void testUploadSuccess() {
        System.out.println("1");
    }

    @Test
    @Order(2)
    public void testDownloadSuccess() {
        System.out.println("2");
    }

    @Test
    @Order(3)
    public void testDeleteSuccess() {
        System.out.println("3");
    }
}

JUnit result:

1
2
3