Parameterized tests¶

Introduction¶

There is often a need to run one unit test several times with different inputs. Such a problem can be solved by duplicating an existing test or extracting a separate method that we call in separate tests. However, such a solution causes the number of unit tests to grow. This problem is solved by parameterized tests, which implement the test code only once. The code can be executed multiple times with different sets of input data.

Tests parameterized in JUnit 5¶

JUnit version 5 provides a set of tools for implementing parameterized tests. For this purpose, the following dependency should be introduced into the project:

<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.6.2</version> <!-- the current version may change -->
    <scope>test</scope> <!-- scope test means that libraries will be visible only in the test bundle -->
</dependency>

@ParameterizedTest¶

Parameterized tests are implemented almost identically to regular unit tests, except for the way they are marked. Instead of the @Test annotation, we use @ParameterizedTest.

Dla poniższej metody statycznej:

public class NumbersHelper {
    public static boolean isOdd(int number) {
        return number % 2 != 0;
    }
}

@ParameterizedTest
@ValueSource(ints = {1, 3, 5, -3, 15, Integer.MAX_VALUE})
void shouldReturnTrueForOddNumbers(int number) {
    assertTrue(NumbersHelper.isOdd(number));
}

The @ValueSource defines the set of inputs that will be passed to the test method as its arguments.

It is the programmer's responsibility to match the number and types of arguments to the source of the arguments. The above example will trigger the unit test six times.

Sources of arguments¶

JUnit offers several ways to define the argument set passed to a parameterized test. They differ in possibilities and the approach to defining parameters. These annotations are:

• @ValueSource
• @EnumSource
• @CsvSource
• @CsvFileSource
• @MethodSource
• @ArgumentsSource

@ValueSource¶

The @ValueSource annotation allows you to pass a single parameter to the test. This parameter can be one of the following types:

NOTE: One limitation of @ValueSource is that it cannot pass asnull.

Below are some examples of parameterization tests (for simplicity with an empty body) using @ValueSource:

class ValueSourceExamplesTest {

  @ParameterizedTest
  @ValueSource(doubles = {1, 2.3, 4.1})
  void shouldPassDoubleToParam(double param) {
  }

  @ParameterizedTest
  @ValueSource(strings = {"Ala", "has a", "cat"})
  void shouldPassStringToTest(String word) {
  }

  @ParameterizedTest
  @ValueSource(classes = {String.class, Integer.class, Double.class})
  void shouldPassClassTypeAsParam(Class<?> clazz) {

  }
}

@EnumSource¶

This annotation allows you to invoke parameterized tests for arguments of enumerated types.

public enum TemperatureConverter {
  CELSIUS_KELVIN(cTemp -> cTemp + 273.15f),
  KELVIN_CELSIUS(kTemp -> kTemp - 273.15f),
  CELSIUS_FAHRENHEIT(cTemp -> cTemp * 9 / 5f + 32);

  private Function<Float, Float> converter;

  TemperatureConverter(Function<Float, Float> converter) {
    this.converter = converter;
  }

  public float convertTemp(float temp) {
    return converter.apply(temp);
  }
}

For the example above, the parameterized tests may look like this. This test will be run for every defined enum value TemperatureConverter.

@ParameterizedTest
@EnumSource(TemperatureConverter.class)
void shouldConvertToValueHigherThanMinInteger(TemperatureConverter converter) {
  assertTrue(converter.convertTemp(10) > Integer.MIN_VALUE);
}

It is also possible to specify specific objects of a given enumeration type using the names attribute, e.g.:

@ParameterizedTest
@EnumSource(value = TemperatureConverter.class, names = {"CELSIUS_KELVIN", "CELSIUS_FAHRENHEIT"})
void shouldConvertToTemperatureLowerThanMaxInteger(TemperatureConverter converter) {
  assertTrue(converter.convertTemp(10) < Integer.MAX_VALUE);
}

Within this annotation, it is also possible to define a mode that determines whether the values given in the names attribute are excluded or included.

The next example shows the use of the EnumSource.Mode.EXCLUDE mode:

@ParameterizedTest
@EnumSource(value = TemperatureConverter.class,
    names = {"KELVIN_CELSIUS"},
    mode = EnumSource.Mode.EXCLUDE)
void shouldConvertTemperatureToPositiveValue(TemperatureConverter converter) {
  assertTrue(converter.convertTemp(10) > 0); // the test will run for the values CELSIUS KELVIN and CELSIUS FAHRENHEIT
}

@CsvSource¶

The @CsvSource annotation allows you to define test parameters using CSV literals ( comma separated value ). Also:

• Data strings are separated by certain characters (commas by default).
• Each separated element is a separate parameter taken in the test.
• This mechanism may be used when we want to provide input parameters and the expected value for the purposes of a given unit test.
• The limit of the @CsvSource annotation is a limited number of types that can be used in the test. All parameter types we want to use in the test must be convertible from the String object.

public class Strings {
  public static String toUpperCase(String input) {
    return input.trim().toUpperCase();
  }
}

@ParameterizedTest
@CsvSource({"  test  ,TEST", "tEst ,TEST", "   Java,JAVA"})
void shouldTrimAndUppercaseInput(String input, String expected) {
  String actualValue = Strings.toUpperCase(input);
  assertEquals(expected, actualValue);
}

The above parameterized test takes text strings as input. In the process of calling the Strings.toUpperCase (input) method the text is stripped of whitespace from the beginning and end of the value and converted to uppercase. It is later compared with the expected value, provied as the second element of the CSV literal .

Within @CsvSource it is possible to change the separator using the attributedelimiter, e.g .:

@ParameterizedTest
@CsvSource(value = {"  test  ;TEST", "tEst ;TEST", "   Java;JAVA"}, delimiter = ';')
void shouldTrimAndUppercaseInput(String input, String expected) {
  String actualValue = Strings.toUpperCase(input);
  assertEquals(expected, actualValue);
}

@CsvFileSource¶

The @CsvFileSource annotation is very similar to the @CsvSource, except the data is loaded directly from the file. It can define the following parameters:

• numLinesToSkip - specifies the number of ignored lines in the source file. This is useful if the data comes from a table that has headers in addition to the value.
• delimeter - means a separator between each element.
• lineSeparator - means the separator between each sets of parameters.
• encoding - means the file content encoding method.

@ParameterizedTest
@CsvFileSource(resources = "/data.csv", numLinesToSkip = 1) // the data.csv file must be in the classpath root, we skip the first line in the file
void shouldUppercaseAndBeEqualToExpected(String input, String expected) {
  String actualValue = Strings.toUpperCase(input);
  assertEquals(expected, actualValue);
}

@MethodSource¶

The @ValueSource,@EnumSource, @CsvSource and @CsvFileSource annotations have limitations - the number of parameters or their types. This problem is solved by the argument source defined by the @MethodSource annotation. The only parameter for this annotation is the * name * of a static method in the same class. This method should be argumentless and return stream:

• objects of any type in the case of tests with a single parameter,
• Arguments objects for tests with multiple parameters.

Both ways of using the @MethodSource annotation show the following examples:

@ParameterizedTest
@MethodSource("provideNumbers")
void shouldBeOdd(final Integer number) {
  assertThat(number % 2).isEqualTo(1);
}

static Stream<Integer> provideNumbers() {
  return Stream.of(1, 13, 101, 11, 121);
}

The next example uses the Arguments.of static method to create a set of arguments for a single parametric test:

@ParameterizedTest
@MethodSource("provideNumbersWithInfoAboutParity")
void shouldReturnExpectedValue(int number, boolean expected) {
  assertEquals(expected, number % 2 == 1);
}

private static Stream<Arguments> provideNumbersWithInfoAboutParity() {
  return Stream.of(Arguments.of(1, true),
      Arguments.of(2, false),
      Arguments.of(10, false),
      Arguments.of(11, true));
}

@ArgumentsSource¶

@ArgumentsSource is the most universal source for defining arguments. Like @MethodSource it allows you to define any number of arguments of any type. The main difference between the two annotations is that inside the @ArgumentsSource annotation we give the type that must implement interface ArgumentsProvider.

The method we need to implement, provideArguments should return a stream of objects of typeArguments.

java.util.stream.Stream<? extends org.junit.jupiter.params.provider.Arguments> provideArguments(org.junit.jupiter.api.extension.ExtensionContext extensionContext) throws java.lang.Exception;

The argument source created this way can be used in many test classes.

public class NumberWithParityArgumentsProvider implements ArgumentsProvider {

  @Override
  public Stream<? extends Arguments> provideArguments(ExtensionContext context) {
    return Stream.of(
        Arguments.of(1, true),
        Arguments.of(100, false),
        Arguments.of(101, true)
    );
  }
}

A sample test using the NumberWithParityArgumentsProvider class might look like this:

@ParameterizedTest
@ArgumentsSource(NumberWithParityArgumentsProvider.class)
void shouldReturnExpectedValue(int number, boolean expectedResult) {
  assertEquals(expectedResult, number % 2 == 1);
}

Connecting sources¶

JUnit allows you to combine multiple argument sources. Because the discussed annotations denoting argument sources do not have the annotation @Repeatable. We can use a specific type of source only once, e.g. the following parameter test will be run 5 times:

class CombinedSourcesTest {

  @ParameterizedTest
  @CsvSource(value = "1, true")
  @MethodSource("provideNumbersWithInfoAboutParity")
  void shouldReturnExpectedValue(int number, boolean expected) {
    assertEquals(expected, number % 2 == 1);
  }

  private static Stream<Arguments> provideNumbersWithInfoAboutParity() {
    return Stream.of(Arguments.of(1, true),
        Arguments.of(2, false),
        Arguments.of(10, false),
        Arguments.of(11, true)
    );
  }
}

Support sources¶

In addition to the standard sources, we have several auxiliary annotations that allow you to add additional parameter values to your existing tests. Those include:

• @NullSource, which allows you to pass the null parameter to the test
• @EmptySource, which allows you to pass an empty object to the test, e.g.
  • "" in case of String class
  • empty collection in case of Collection
  • empty array when using array.
• @NullAndEmptySource, which combines the functionality of the @NullSource and @EmptySource annotations.

The following examples show their use:

public class Strings {
    public static boolean isBlank(String input) {
        return input == null || input.trim().isEmpty();
    }
}

@ParameterizedTest
@NullSource
void shouldbeBlankForNull(String input) {
  assertTrue(Strings.isBlank(input));
}

public class Arrays {
  public static boolean isValid(List<String> values) {
    return values != null && !values.isEmpty();
  }
}

@ParameterizedTest
@EmptySource
void shouldNotBeValid(List<String> input) {
  assertFalse(Arrays.isValid(input));
}

@ParameterizedTest
@NullAndEmptySource
void nullAndEmptyShouldBeBlank(String input) {
  assertTrue(Strings.isBlank(input));
}