Migrating Selenium , Java Based JUnit 4 Automation Tests to JUnit 5

9 min readJun 28, 2021

The objective of this post is to explain and share my experience migrating existing JUnit 4 Selenium, Java based Automation Tests or framework to JUnit 5.

Our Story: It’s been 7 years we are running JUnit4 based automation tests(around 20,000+ regression tests for 20 + web applications in Dotdash) We also have well designed automation framework built on top of Selenium, Java based which supports for running all tests based on different browsers, devices, environments, operating systems and different test categories(smoke, regression, visual etc..). we run Automation Tests using docker containerization way . checkout my blog here Setting up an on-demand Selenium Grid with containers.

JUnit 4 Current Setup:

There are two ways we run tests.

1. Using IDE’s (Eclipse, IntelliJ etc..)
2. Running through command line(just like running Java Application)by Jenkins pipeline with remote Selenium Grid.

JUnit 5 Overview:

What is JUnit 5?

JUnit 5 is a powerful and flexible update to the JUnit framework, and it provides a variety of improvements and new features to organize and describe test cases
Unlike previous versions of JUnit, JUnit 5 is composed of several different modules from three different sub-projects.
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
JUnit Platform serves as a foundation for launching testing frameworks on the JVM.
JUnit Jupiter is the combination of the new programming model and extension model for writing tests and extensions in JUnit 5. The Jupiter sub-project provides a TestEngine for running Jupiter based tests on the platform.
JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on the platform.

JUnit 5 Architecture & Maven Dependency:

please check all Junit5 Maven dependencies available from maven repository https://mvnrepository.com/artifact/org.junit.jupiter .

JUnit 5 Features:

JUnit 5 provided a lot of new features to write and run tests along with existing JUnit 4 features. I am not going to talk about each one but I have included some of the features in the sample code and explained differences below. To understand all of these features and changes for some of the JUnit4 Annotations, Assertions, Assumptions please go through the JUnit 5 user guide https://junit.org/junit5/docs/current/user-guide/ or refer sample code.

Methods for asserting reside in the org.junit.jupiter.api.Assertions class instead of org.junit.Assert class.
Assumption methods reside in org.junit.jupiter.Assumptions class instead of org.junit.Assume class.
The @Category annotation from JUnit 4 has been replaced with a @Tag annotation in JUnit 5. Also, we no longer use marker interfaces but instead pass the annotation a string parameter or create Custom tags and use them. for example,

4. Custom Tags: you can create custom Tags using @Tag and use for Tests. for example to create SmokeTests Tag and use later for the tests,

5. you can also include @Test with custom Tag and directly use custom Tag(for example @RegressionTest) for Test. for example,

6. In JUnit 4 writing parameterized tests required using a Parameterized runner. In addition, we needed to provide parameterized data via a method annotated with the @Parameterized.Parameters annotation.

7. Annotations : below are some changes made for existing JUnit4 annotations.

Conditional Test execution:

JUnit 5 provides the ExecutionCondition extension API to enable or disable a test or container (test class) conditionally. This is like using @Disabled on a test but it can define custom conditions. There are multiple built-in conditions, such as,

@EnabledOnOs and @DisabledOnOs: Enables or disables a test only on specified operating systems.

@EnabledOnJre and @DisabledOnJre: Specifies the test should be enabled or disabled for particular versions of Java.

@EnabledIfSystemProperty: Enables a test based on the value of a JVM system property.

@EnabledIf: Uses scripted logic to enable a test if scripted conditions are met.

@EnabledIfEnvironmentVariable and @DisabledIfEnvironmentVariable : enabled or disabled based on the value of the named environment variable.

Custom conditions : @EnabledIf(“customCondition”) & @DisabledIf(“customCondition”)

JUnit 5 Features: Advanced:

These are the more interesting topics/features to discover and run tests. before migrating to JUnit5 , we were running JUnit 4 tests by identifying/scanning test classes at run time using Java reflection but now using theses JUnit 5 advanced features, we get rid of all old logic/code and it’s simple and easy way to understand, write code.

JUnit Platform Launcher API : JUnit 5 introduces the concept of a Launcher that can be used to discover, filter, and execute tests. The launcher API is in the junit-platform-launcher module.
Discovering Tests : Introducing test discovery as a dedicated feature of the platform itself will (hopefully) free IDEs and build tools from most of the difficulties they had to go through to identify test classes and test methods in the past.
Executing Tests : To execute tests, clients can use the same LauncherDiscoveryRequest as in the discovery phase or create a new request. Test progress and reporting can be achieved by registering one or more TestExecutionListener implementations with the Launcher.

4. Here is sample example code in main() method to run regression tests, smoke tests and single tests from your project.

Note: XmlReportGeneratingListener used in main() method is a listener which extending TestExecutionListener to log events for each test and generate test reports.you can use your own listener or JUnit5 provided TestExecutionListener. I am not explaining more details about it.

Parallel Execution:

Note: Very Very Important topic.please read , understand and practice.

By default, JUnit Jupiter tests are run sequentially in a single thread. Running tests in parallel, e.g. to speed up execution, is available as an opt-in feature since version 5.3.We can configure parallel tests execution for JUnit 5 tests in several ways,

By providing parameters to maven surefire plugin. You can set JUnit Platform configuration parameters to influence test discovery and execution by declaring the configurationParameters property and providing key-value pairs using the Java Properties file syntax (as shown below). please add or delete configuration according to your requirements.

mavn-surefire-plugin with JUnit5 parallel configuration

2. By providing as System properties to JVM: In Java, you can set system properties either from the command line or from the application code itself. For example, to run only test methods in the org.example.MyTest test class you can execute mvn -Dtest=org.example.MyTest test from the command line or you can set property from the application like , System.setProperty(“test” , “org.example.MyTest”);

3. The JUnit Platform configuration file: The simplest way is include parameter junit.jupiter.execution.parallel.enabled=true in junit-platform.properties file (src/main/resources).

Configuration to execute top-level classes in parallel but methods in same thread.

Note: You will notice from the out put that, each group of tests started by one different thread and each group test methods started by same thread. try to run the tests and check the console log to see differences.
strategies take different approaches to configure the environment of the parallel execution, such as the number of threads to use. If no configuration strategy is set, JUnit Jupiter uses the dynamic configuration strategy with a factor of 1, i.e. the desired parallelism will equal the number of available processors/cores.

The fixed strategy:

The fixed strategy uses a hardcoded number of threads to drive parallel execution.To set this strategy, use configuration parameter junit.jupiter.execution.parallel.config.strategy=fixed
To configure fixed strategy, use configuration parameter junit.jupiter.execution.parallel.config.fixed.parallelism and set it to an int value.

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = same_thread
junit.jupiter.execution.parallel.mode.classes.default = concurrent
junit.jupiter.execution.parallel.config.strategy = fixed
junit.jupiter.execution.parallel.config.fixed.parallelism = 10
junit.platform.output.capture.stdout = true
junit.platform.output.capture.stderr = true

Configuration to execute top-level classes in sequentially but their methods in parallel.

Note: You will notice from the out put that, each test class started sequentially and test methods are executed in parallel by different threads.

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = same_thread
junit.jupiter.execution.parallel.config.strategy = fixed
junit.jupiter.execution.parallel.config.fixed.parallelism = 10
junit.platform.output.capture.stdout = true
junit.platform.output.capture.stderr = true

Dynamic strategy:

Dynamic is the default parallel execution strategy.
It utilizes the number of CPU cores to determine parallelism.
To set this strategy explicitly, use configuration parameter junit.jupiter.execution.parallel.config.strategy=dynamic
To scale this value, use configuration parameter junit.jupiter.execution.parallel.config.dynamic.factor and set it to an int value (default: 1).

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.mode.default = concurrent
junit.jupiter.execution.parallel.mode.classes.default = same_thread
junit.jupiter.execution.parallel.config.strategy=dynamic
junit.jupiter.execution.parallel.config.dynamic.factor = 10
junit.platform.output.capture.stdout = true
junit.platform.output.capture.stderr = true

Custom strategy:

For developers that seek complete control over their concurrent execution, the implementation of a ParallelExecutionConfiguration is exposed to the TestEngine. This also provides a mechanism to consume custom configuration parameters passed to the JVM through the public API of the ParallelExecutionConfigurationStrategy class.
To write a custom execution strategy, a dependency on the junit-platform-engine library is required inside the test project.
To set this strategy explicitly, use configuration parameter . junit.jupiter.execution.parallel.config.strategy = custom
Use configuration parameter junit.jupiter.execution.parallel.config.custom.class and set it to the implementation class.
If the strategy requires more external data, it can define its own configuration parameters. The name is arbitrary, but it must start with the "junit.jupiter.execution.parallel.config" prefix. below is sample code.

implementing ParallelExecution Configuration , Strategy

Capturing Standard Output/Error:

Since version 1.3, the JUnit Platform provides opt-in support for capturing output printed to System.out and System.err. To enable it, simply set the junit.platform.output.capture.stdout and/or junit.platform.output.capture.stderr configuration parameter to true.

Extension Model

The @Rule and @ClassRule annotations from JUnit 4 do not exist in JUnit 5. We can implement the same functionality by using the new extension model in the org.junit.jupiter.api.extension package and the @ExtendWith annotation.

In contrast to the competing Runner, @Rule, and @ClassRule extension points in JUnit 4, the JUnit Jupiter extension model consists of a single, coherent concept: the Extension API. Developers can register one or more extensions declaratively by annotating a test interface, test class, test method, or custom composed annotation with @ExtendWith(…) and supplying class references for the extensions to register.

we have a requirement that need to post/delete some document Json data to document database before/after running tests. we were using @Rule in JUnit4 . now we have achieved same with implementing BeforeEachCallback , AfterEachCallback and registering extensions declaratively by annotating a test class/test with @ExtendWith.

@ExtendWith is a repeatable annotation that is used to register extensions for the annotated test class or test method. for example if you want to implement @ExtendWith at class level.

we can also use @ExtendWith at test level, for example,

ErrorCollector:

The ErrorCollector rule allows execution of a test to continue after the first problem is found (for example, to collect _all_ the incorrect rows in a table, and report them all at once). we have been using ErrorCollector with hamcrest matchers to report test failures. example code is here in my test method,

JUnit5 has provided some solution to migrate/convert test assertions. here is example ,

you can find all JUnit5 Assertions over here

But after moving to Junit5, we could not able to migrate all test assertions using the above provided solution and we thought that it involves lot of code refactoring .

However, to provide a gradual migration path there is support for a subset of JUnit 4 rules and their subclasses in junit-jupiter-migrationsupport module.

Existing code using these rules can be left unchanged by using the class level annotation @EnableRuleMigrationSupport in the org.junit.jupiter.migrationsupport.rules package.

To enable the support in Maven we have to add the below dependency in pom.xml.

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-migrationsupport</artifactId>
        <version>5.7.0</version>
    </dependency>
</dependencies>

here is example how to use @EnableRuleMigrationSupport at class level.

what else? there are many features , but I am concluding here.

References:

https://junit.org/junit5/docs/current/user-guide/

About Me:

This is Mahesh Oruganti working as Automation QA Engineer at dotDash. I hope you enjoy this Article. Happy learning. Thanks!!