Splitting tests using JUnit Tags

Summary 📖

As a project scales, there will (should) be more tests and in a perfect world, these new tests should not have a noticeable change to the build times of your application. But the world is not perfect, developers are lazy, and after a short time your build times have ballooned 🎈

Common ideology in software circles is the concept of the testing pyramid, but if you find yourself working on a project that has an inverted testing pyramid (that is, a greater proportion of integration-style testing) - you probably have slow build pipelines. A colleague once referred to this style of testing as the testing “funeral urn” ⚱️, which is rather fitting to how you might feel waiting over 2 hours to get build results. 🥲

It’s not feasible to re-implement tens of thousands of tests overnight, so grouping together tests can help you to parallelize your integration tests and make your build times faster (you’re gonna need a bigger build box).

Who might find this useful? 🤔

If any of the below apply to you - you might be in the right place. Don’t worry, I won’t judge you - nor will I tell you how many apply to my projects…

1. Have you got a slow build?
2. Have you got a lot of slow tests?
3. Are you using Spock and/or Junit 5?
4. Are you currently using JUnit categories to group your tests, but are trying to upgrade to newer flavours of Spock and/or JUnit?
5. Are you trying to get to Java 17 which is causing your to also upgrade Gradle, Groovy, and Spock and are you losing your mind and regretting how many tests you have written in Spock?

Ok fine, they all apply to me.

Tag, you’re it 🏷️

JUnit 5 added tag support - which provides a way to categorize tests and later filter them based on some tag expressions. Tags are a full replacement of categories, which required a lot of boilerplate code to implement - with tags, an annotation does the trick;

Copy 
1import org.junit.jupiter.api.Assertions;
2import org.junit.jupiter.api.Tag;
3import org.junit.jupiter.params.ParameterizedTest;
4import org.junit.jupiter.params.provider.CsvSource;
5
6@Tag("JUnit")
7public class DivideJunitTests {
8
9    @Tag("Divide")
10    @ParameterizedTest
11    @CsvSource({"10,10,1", "10,1,10", "4200,10,420"})
12    public void testMultiply(int a, int b, int expected) {
13        Assertions.assertEquals(expected, Divide.divide(a, b));
14    }
15}

The idea then is that you can scatter these tags across your test suites - and speed up your builds by running each of the tags in parallel 💨

And the good news for those who are using a hybrid cocktail Spock and JUnit is that Spock uses a JUnit runner under the hood. Spock 2.3 introduced its own tags which play nicely with the JUnit variant.

Code 🧑‍💻

All the code in this post can be found on my GitHub 🖖

Disclaimer: it’s generally not advised to duplicate all of your tests in both Java and Groovy like these examples - that most likely leads to less than optimal build time, too.

Suppose you have 2 test classes - one in JUnit;

Copy 
1import org.junit.jupiter.api.Assertions;
2import org.junit.jupiter.api.Tag;
3import org.junit.jupiter.params.ParameterizedTest;
4import org.junit.jupiter.params.provider.CsvSource;
5
6@Tag("JUnit")
7public class DivideJunitTests {
8
9    @Tag("Divide")
10    @ParameterizedTest
11    @CsvSource({"10,10,1", "10,1,10", "4200,10,420"})
12    public void testDivide(int a, int b, int expected) {
13        Assertions.assertEquals(expected, Divide.divide(a, b));
14    }
15}

..and one in Groovy;

Copy 
1import spock.lang.Specification
2import spock.lang.Tag
3
4@Tag("Spock")
5class DivideSpockTests extends Specification {
6
7    @Tag("Divide")
8    void "Test divide"(int a, int b, int expected) {
9        expect:
10            Divide.divide(a, b) == expected
11        where:
12            a    | b  | expected
13            10   | 10 | 1
14            10   | 1  | 10
15            4200 | 10 | 420
16    }
17}

As expected, these can both be ran via Gradle;

Copy 
1$ ./gradlew test
2
3> Task :test
4
5DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 10, b: 10, expected: 1, #0] PASSED
6
7DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 10, b: 1, expected: 10, #1] PASSED
8
9DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 4200, b: 10, expected: 420, #2] PASSED
10
11DivideJunitTests > testDivide(int, int, int) > com.mtjb.demo.math.DivideJunitTests.testDivide(int, int, int)[1] PASSED
12
13DivideJunitTests > testDivide(int, int, int) > com.mtjb.demo.math.DivideJunitTests.testDivide(int, int, int)[2] PASSED
14
15DivideJunitTests > testDivide(int, int, int) > com.mtjb.demo.math.DivideJunitTests.testDivide(int, int, int)[3] PASSED
16
17BUILD SUCCESSFUL in 3s
185 actionable tasks: 5 executed

With a few tweaks to your build file;

Copy 
1test {
2    useJUnitPlatform {
3
4        def groups = System.getProperty("groups")
5        println(groups)
6        if (groups != null) {
7            includeTags(groups)
8        }
9    }
10    testLogging {
11        events "passed", "skipped", "failed"
12    }
13}

You can start to filter what tests are being ran;

Copy 
1$ ./gradlew test -Dgroups=Spock
2
3> Task :test
4
5DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 10, b: 10, expected: 1, #0] PASSED
6
7DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 10, b: 1, expected: 10, #1] PASSED
8
9DivideSpockTests > Test divide > com.mtjb.demo.math.DivideSpockTests.Test divide [a: 4200, b: 10, expected: 420, #2] PASSED
10
11BUILD SUCCESSFUL in 3s
125 actionable tasks: 5 executed

⚡️ Conclusion

JUnit tags, unsurprisingly, works as expected! If you’re like me, and are working on a project with a web of dependencies you might find this harder than it ought to be - and for that I advise some self-reflection to see ifyou can figure out why you’ve written such confusing build files (or if you’re like me, to curse that guy who used to work here!).