Cédric Champeau's blog: Introducing Micronaut Test Resources

Test resources in a nutshell

Micronaut Test Resources simplifies testing of applications which depend on external resources, by handling the provisioning and lifecycle of such resources automatically. For example, if your application requires a MySQL server, in order to test the application, you need a MySQL database to be installed and configured, which includes a database name, a username and a password. In general, those are only relevant for production, where they are fixed. During development, all you care about is having one database available.

Here are a couple of traditional solutions to this problem:

document that a MySQL server is a pre-requisite, and give instructions about the database to create, credentials, etc. This can be simplified by using Docker containers, but there’s still manual setup involved.
Use a library like Testcontainers in order to simplify the setup

In general, using Testcontainers is preferred, because it integrates well with the JVM and provides an API which can be used in tests to spawn containers and interact with them. However, a better integration between Micronaut and Testcontainers can improve the developer experience in several ways:

simplify the container lifecycle configuration by providing an opinionated framework-specific default way, making you think less of how to setup it in the individual tests : tests shouldn’t need to deal with the container lifecycle: we’d like to have test containers/resources management as transparent as possible.
isolate it better from your application making it simpler to reason about dependencies (and transitive dependencies), not just for the developer, but for example tools enabling native mode as well: Testcontainers APIs "leak" to the test classpath, making it difficult to run tests in native mode. This is not a problem specific to the Testcontainers library though: many libraries are not yet compatible with GraalVM. Our solution makes it possible to use Testcontainers in native tests without the hassle of configuring it!
enable support for "development mode", that is to say when you run the application locally (not the tests, the application itself) or even several distinct projects at once that can benefit from sharing access to the same running containers (for example, an MQTT client and a server may want to use the same container, even if they are individual projects living in distinct Git repositories).

The goal of Micronaut Test Resources is to achieve all of these at once:

zero-configuration: without adding any configuration, test resources should be spawned and the application configured to use them. Configuration is only required for advanced use cases.
classpath isolation: use of test resources shouldn’t leak into your application classpath, nor your test classpath
compatible with GraalVM native: if you build a native binary, or run tests in native mode, test resources should be available
easy to use: the Micronaut build plugins for Gradle and Maven should handle the complexity of figuring out the dependencies for you
extensible: you can implement your own test resources, in case the built-in ones do not cover your use case
technology agnostic: while lots of test resources use Testcontainers under the hood, you can use any other technology to create resources

In addition, Micronaut Test Resources support advanced development patterns, which are useful in the microservices era. As an example, it is capable of sharing containers between submodules of a single build, or even between independent projects, from different Git repositories! Say that you have 2 projects, one built with Gradle, the other with Maven, both needing to communicate using the same message bus: Micronaut is capable of handling this use case for you, making it extremely easy to test components interacting with each other!

Because of these constraints, we decided to use Testcontainers, because the library is just perfect for the job, but in an isolated process instead, as I’m going to describe below. Note that this solution is also 100% compatible with Testcontainers Cloud, which makes container provisioning even easier!

Using Micronaut Test Resources

Enabling test resources support

Micronaut Test Resources integrates with build tools. In both Maven and Gradle, you need to enable test resources support. If you create a new project using Micronaut Launch or the Micronaut CLI, test resources will be configured for you, but if you migrate an existing application to test resources, here’s what you need to do:

If you are using Maven, you will need to upgrade to the Micronaut 3.6 parent POM and add the following property:

<properties>
   <micronaut.test.resources.enabled>true</micronaut.test.resources.enabled>
</properties>

For Gradle, you can use test resources with Micronaut 3.5+ and you simply need to use the test resources plugin:

plugins {
    id 'io.micronaut.application' version '3.5.1'
    id 'io.micronaut.test-resources' version '3.5.1'
}

Our first test resources contact

In this blog post we will write an application which makes use of Micronaut Data and connects to a MySQL server to list books. The whole application code is available on GitHub, so I’m only going to show the relevant parts for clarity.

In such an application, we typically need a repository:

@JdbcRepository(dialect = Dialect.MYSQL)
public interface BookRepository extends CrudRepository<Book, Long> {
    @Override
    List<Book> findAll();
}

And this repository makes use of the Book class:

@MappedEntity
public class Book {
    @Id
    @GeneratedValue(GeneratedValue.Type.AUTO)
    private Long id;

    private String title;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }
}

In order for Micronaut to use the database, we need to add some configuration to our application.yml file:

datasources:
  default:
    schema-generate: CREATE
    db-type: mysql

The most important thing to see is that we don’t specify any username, password or URL to connect to our database: the only thing we have to specify is the database type of our datasource. We can then write the following test:

@MicronautTest
class DemoTest {

    @Inject
    BookRepository bookRepository;

    @Test
    @DisplayName("A MySQL test container is required to run this test")
    void testItWorks() {
        Book book = new Book();
        book.setTitle("Yet Another Book " + UUID.randomUUID());
        Book saved = bookRepository.save(book);
        assertNotNull(saved.getId());
        List<Book> books = bookRepository.findAll();
        assertEquals(1, books.size());
    }

}

The test creates a new book, stores it in the database, then checks that we get the expected number of books when reading the repository. Note, again, that we didn’t have to specify any container whatsoever. In this blog post I’m using Gradle, so we can verify the behavior by running:

./gradlew test

Then you will see the following output (cleaned up for clarity of this blog post):

i.m.testresources.server.Application - A Micronaut Test Resources server is listening on port 46739, started in 128ms
i.m.t.e.TestResourcesResolverLoader - Loaded 2 test resources resolvers: io.micronaut.testresources.mysql.MySQLTestResourceProvider, io.micronaut.testresources.testcontainers.GenericTestContainerProvidereted
o.testcontainers.DockerClientFactory - Connected to docker:
  Server Version: 20.10.17
  API Version: 1.41
  Operating System: Linux Mint 20.3
  Total Memory: 31308 MB
🐳 [testcontainers/ryuk:0.3.3] - Creating container for image: testcontainers/ryuk:0.3.3
🐳 [testcontainers/ryuk:0.3.3] - Container testcontainers/ryuk:0.3.3 is starting: 1f5286fa728aca74a7d6d4c0eb2148a3bc81f5c028027496d7aabda7b7ed45e8
🐳 [testcontainers/ryuk:0.3.3] - Container testcontainers/ryuk:0.3.3 started in PT0.655476S
o.t.utility.RyukResourceReaper - Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
🐳 [mysql:latest] - Creating container for image: mysql:latest
🐳 [mysql:latest] - Container mysql:latest is starting: d796c7a1ce10f393a4181f12967ee77ac9864f45595f97967c700f022e86ac7d
🐳 [mysql:latest] - Waiting for database connection to become available at jdbc:mysql://localhost:49209/test using query 'SELECT 1'
🐳 [mysql:latest] - Container is started (JDBC URL: jdbc:mysql://localhost:49209/test)
🐳 [mysql:latest] - Container mysql:latest started in PT7.573915S

BUILD SUCCESSFUL in 11s
7 actionable tasks: 2 executed, 5 up-to-date

What does this tell us? First, that a "Micronaut Test Resources server" was spawned, for the lifetime of the build. When the test was executed, this service was used to start a MySQL test container, which was then used during tests. We didn’t have to configure anything, test resources did it for us!

Running the application

What is also interesting is that this also works if you run the application in development mode. Using Gradle, you do this by invoking ./gradlew run (mvn mn:run with Maven): as soon as a bean requires access to the database, a container will be spawned, and automatically shut down when you stop the application.

Note	Of course, in production, there won’t be any server automatically spawned for you: Micronaut will rely on whatever you have configured, for example in an `application-prod.yml` file. In particular, the URL and credentials to use.

What is even nicer is that you can use this in combination with Gradle’s continuous mode!

To illustrate this, let’s create a controller for our books:

@Controller("/")
public class BookController {
    private final BookRepository bookRepository;

    public BookController(BookRepository bookRepository) {
        this.bookRepository = bookRepository;
    }

    @Get("/books")
    public List<Book> list() {
        return bookRepository.findAll();
    }

    @Get("/books/{id}")
    public Book get(Long id) {
        return bookRepository.findById(id).orElse(null);
    }

    @Delete("/books/{id}")
    public void delete(Long id) {
        bookRepository.deleteById(id);
    }
}

Now start the application in continuous mode: ./gradlew -t run

You will see that the application starts a container as expected:

INFO  io.micronaut.runtime.Micronaut - Startup completed in 9166ms. Server Running: http://localhost:8080

Notice how it took about 10 seconds to start the application, most it it spent in starting the MySQL test container itself. You definitely don’t want to pay this price for every change you make, so this is where the continuous mode is helpful. If we ask for the list of books, we’ll get an empty list:

$ http :8080/books
HTTP/1.1 200 OK
Content-Type: application/json
connection: keep-alive
content-length: 2
date: Tue, 26 Jul 2022 16:59:51 GMT

[]

This is expected, but notice how we didn’t have a method to actually add a book to our store. Let’s fix this by editing the BookController.java class without stopping the server. Add the following method:

    @Get("/books/add/{title}")
    public Book add(String title) {
        Book book = new Book();
        book.setTitle(title);
        return bookRepository.save(book);
    }

Save the file and notice how Gradle instantly reloads the application, but doesn’t restart the database: it’s already there so it’s going to reuse it!

In the logs you will see something like this:

INFO  io.micronaut.runtime.Micronaut - Startup completed in 1086ms. Server Running: http://localhost:8080

This time the application started in just a second! Let’s add a book:

$ http :8080/books/add/Micronaut%20in%20action
HTTP/1.1 200 OK
Content-Type: application/json
connection: keep-alive
content-length: 38
date: Tue, 26 Jul 2022 17:03:57 GMT

{
    "id": 1,
    "title": "Micronaut in action"
}

However, if we stop the application (by hitting CTRL+C) and start again, you will see that the database will be destroyed when the application shuts down. Let’s see how we can "survive" different build invocations.

Keeping the service alive

By default, the test resources service is short lived: it’s going to be started at the beginning of a build, and shutdown at the end of a build. This means, that it will live as long as you have tests running, or, if running in development mode, as long as the application is alive. However, you can make it survive the build, and reuse the containers in several, independent build invocations.

To do this, you need to explicitly start the test resources service:

./gradlew startTestResourcesService

This starts the test resources service in the background: it did not start our application, nor did it run tests. This means that now, we can start our application:

./gradlew run

And, because it’s the first time the application is launched since we started the test resources service, it’s going to spawn a test container:

INFO  io.micronaut.runtime.Micronaut - Startup completed in 9211ms. Server Running: http://localhost:8080

We can add our book:

$ http :8080/books/add/Micronaut%20in%20action
HTTP/1.1 200 OK
Content-Type: application/json
connection: keep-alive
content-length: 38
date: Tue, 26 Jul 2022 17:03:57 GMT

{
    "id": 1,
    "title": "Micronaut in action"
}

The difference is now that if we stop the application (e.g hit CTRL+C) and start it again, it will reuse the container:

INFO  io.micronaut.runtime.Micronaut - Startup completed in 895ms. Server Running: http://localhost:8080

If we list our books, the database wasn’t cleaned, so we’ll get the book we created from the previous time we started the app:

$ http :8080/books
HTTP/1.1 200 OK
Content-Type: application/json
connection: keep-alive
content-length: 40
date: Tue, 26 Jul 2022 17:14:40 GMT

[
    {
        "id": 1,
        "title": "Micronaut in action"
    }
]

Nice, right? However there’s a gotcha if you do this: what happens if we run tests?

$ ./gradlew test

> Task :compileTestJava
Note: Creating bean classes for 1 type elements

> Task :test FAILED

DemoTest > A MySQL test container is required to run this test FAILED
    org.opentest4j.AssertionFailedError at DemoTest.java:28

Why is that? This is simply because our tests expect a clean database, and we had a book in it, so keep this in mind if you’re using this mode.

At some point, you will want to close all open resources. You can do this by explicitly stopping the test resources service:

./gradlew stopTestResourcesService

Now, you can run the tests again and see them pass:

$ ./gradlew test

...
INFO  🐳 [testcontainers/ryuk:0.3.3] - Creating container for image: testcontainers/ryuk:0.3.3
INFO  🐳 [testcontainers/ryuk:0.3.3] - Container testcontainers/ryuk:0.3.3 is starting: ea2aa1c7f1e66a9c7306b00443e8a6693451f3f02bd780b3e2ed7b96ed59936a
INFO  🐳 [testcontainers/ryuk:0.3.3] - Container testcontainers/ryuk:0.3.3 started in PT0.553559699S
INFO  o.t.utility.RyukResourceReaper - Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
INFO  o.testcontainers.DockerClientFactory - Checking the system...
INFO  o.testcontainers.DockerClientFactory - ✔︎ Docker server version should be at least 1.6.0
INFO  🐳 [mysql:latest] - Creating container for image: mysql:latest
INFO  🐳 [mysql:latest] - Container mysql:latest is starting: 1c6437a55b8f9e5668bcec4aef27087c889b8a77ca18d2ddf58809853482a422
INFO  🐳 [mysql:latest] - Waiting for database connection to become available at jdbc:mysql://localhost:49227/test using query 'SELECT 1'
INFO  🐳 [mysql:latest] - Container is started (JDBC URL: jdbc:mysql://localhost:49227/test)
INFO  🐳 [mysql:latest] - Container mysql:latest started in PT7.469460173S

BUILD SUCCESSFUL in 11s
7 actionable tasks: 2 executed, 5 up-to-date

Native testing

Did you know that you can run your test suite in native mode? That is to say, that the test suite is going to be compiled into a native binary which runs tests? One issue with this approach is that it’s extremely complicated to make it work with Testcontainers, as it requires additional configuration. With Micronaut Test Resources, there is no such problem: you can simply invoke ./gradlew nativeTest and the tests will properly run. This works because Testcontainers libraries do not leak into your test classpath: the process which is responsible for managing the lifecycle of test resources is isolated from your tests!

Under the hood

How does that work?

In a nutshell, Micronaut is capable of reacting to the absence of a configured property. For example, a datasource, in order to be created, would need the value of the datasources.default.url property to be set. Micronaut Test Resources work by injecting those properties at runtime: when the property is read, it triggers the creation of test resources. For example, we can start a MySQL server, then inject the value of the JDBC url to the datasources.default.url property. This means that in order for test resources to work, you need to remove configuration (note that for production, you will need to provide an additional configuration file, for example application-prod.yml, which provides the actual values).

The entity which is responsible for resolving missing properties is the "Test Resources Server": it’s a long lived process which is independent from your application and it is responsible for managing the lifecycle of test resources. Because it’s independent from the application, it means it can contain dependencies which are not required in your application such as, typically, the Testcontainers runtime. But it may also contain additional classes, like JDBC drivers, or even your custom test resources resolver!

Because this test resources server is a separate process, it also means it can be shared by different applications, which is the reason why we can share the same containers between independent projects.

Once you understand that Micronaut Test Resources work by resolving missing properties, it becomes straightforward to configure. In particular, we offer configuration which makes it very easy to support scenarios which are not supported out of the box. For example, Micronaut Test Resources supports several JDBC or reactive databases (MySQL, PostgreSQL, MariaDB, SQL Server and Oracle XE), Kafka, Neo4j, MQTT, RabbitMQ, Redis, Hashicorp Vault and ElasticSearch, but what if you need a different container?

In that case, Micronaut Test Resources offer a conventional way to create such containers, by simply adding some configuration lines: in the documentation we demonstrate how to use the fakesmtp SMTP server with Micronaut Email for example.

Custom test resources

If the configuration-based support isn’t sufficient, you also have, in addition, the ability to write your own test resources. If you use Gradle, which I of course recommend, this is made extremely easy by the test resources plugin, which creates an additional source set for this, named testResources. For Maven, you would have to create an independent project manually to support this scenario.

As an illustration, let’s imagine that we have a bean which reads a configuration property:

@Singleton
public class Greeter {
     private final String name;

     public Greeter(@Value("${my.user.name}") String name) {
         this.name = name;
     }

     public String getGreeting() {
     	return "Hello, " + name + "!";
     }

     public void sayHello() {
         System.out.println(getGreeting());
     }
}

This bean requires the my.user.name property to be set. We could of course set it in an application-test.yml file, but for the sake of the exercise, let’s imagine that this value is dynamic and needs to be read from an external service. We will implement a custom test resources resolver for this purpose.

Let’s create the src/testResources/java/demo/MyTestResource.java file with the following contents:

package demo;

import io.micronaut.testresources.core.TestResourcesResolver;

import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Optional;

public class MyTestResource implements TestResourcesResolver {

    public static final String MY_TEST_PROPERTY = "my.user.name";

    @Override
    public List<String> getResolvableProperties(Map<String, Collection<String>> propertyEntries, Map<String, Object> testResourcesConfig) {
        return Collections.singletonList(MY_TEST_PROPERTY); // (1)
    }

    @Override
    public Optional<String> resolve(String propertyName, Map<String, Object> properties, Map<String, Object> testResourcesConfiguration) {
        if (MY_TEST_PROPERTY.equals(propertyName)) {
            return Optional.of("world");                    // (2)
        }
        return Optional.empty();
    }

}

Tells that this resolver can resolve the my.user.name property
Returns the value of the my.user.name property

And in order for the resolver to be discovered, we need to create the src/testResources/resources/META-INF/services/io.micronaut.testresources.core.TestResourcesResolver file with the following contents:

demo.MyTestResource

Now let’s write a test for this by adding the src/test/java/demo/GreeterTest.java file:

package demo;

import io.micronaut.context.annotation.Requires;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import jakarta.inject.Inject;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

@MicronautTest
class GreeterTest {

    @Inject
    Greeter greeter;


    @Test
    @DisplayName("Says hello")
    void saysHello() {
        greeter.sayHello();
        Assertions.assertEquals("Hello, world!", greeter.getGreeting());
    }

}

Now if you run ./gradlew test, you will notice that Gradle will compile your custom test resource resolver, and when the test starts, you will read the following line:

Loaded 3 test resources resolvers: demo.MyTestResource, io.micronaut.testresources.mysql.MySQLTestResourceProvider, io.micronaut.testresources.testcontainers.GenericTestContainerProvider

So when the Greeter bean is created, it will read the value of the my.user.name property by calling your custom test resolver! Of course this is a very simple example, and I recommend that you take a look at the Micronaut Test Resources sources for more examples of implementing resolvers.

Conclusion

In this blog post, we’ve explored the new Micronaut Test Resources module, which will greatly simplify development of Micronaut applications which depend on external services like databases or messaging queues. It works by simplifying configuration, by removing lines which used to be present, but now are dynamically resolved, like datasources.default.url. Test resources are handled in a separate process, the test resources server, which is responsible for handling their lifecycle. This also makes it possible to share the resources (containers, databases, …) between independent builds. For advanced use cases, Micronaut Test Resources provides configuration based resources creation.

Last but not least, Micronaut Test Resources is an extensible framework which will let you implement your own test resources in case the built-in ones miss a feature.

Special thanks to Tim Yates for his hard work on upgrading the Micronaut Guides to use test resources, and Álvaro Sanchez-Mariscal for his support on the Maven plugin!