The acceptable way to integration tests with Spring and Kotlin

When I started to implement tests with Spring Boot, I struggled to find a way to implement integration tests.

Integration tests need to pass through all the flow, beginning at the controller layer and ending at the database layer. But how could I access the database to perform a test? Here are some options:

  1. Access a different database that exists only for test purposes. This option requires a database infrastructure to maintain, resulting in more costs.
  2. Use the @DataJpaTest to instance an H2 memory database, which will die by the end of the test. But it doesn’t really test in all ways your database layer, because it doesn’t use the same drivers and keywords of your production database. For example: If you have a table named User, with H2 it will run perfectly, but with PostgreSQL, it will fail, since User is a keyword.
  3. Use Testcontainers to build a Docker container with the same database you use in production. Yes, that’s the best option.

What is Testcontainers?

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
Testcontainers Docs

But what does that mean?
It means that you can run not only databases but everything that can run in a Docker container! You can test the whole flow of your app, passing through databases (like PostgreSQL, MySQL, etc…), cache strategies (like using Redis), and more!

How to

To demonstrate how to use Testcontainers I created a simple app with Spring Initializr using Kotlin and Gradle:

  • Spring Web to develop endpoints
  • Spring Data JPA to access the database without creating native queries
  • PostgreSQL Driver to connect to the database

This is how our app will look like:

The base project

This simple app has a Book CRUD with this structure:

The Model:

@Entity
data class Book(
    @Id
    val id: UUID = UUID.randomUUID(),
    val title: String? = null,
    val author: String? = null
)

The Controller:

@RestController
@RequestMapping("/books")
class BookController(private val bookService: BookService) {

    @PostMapping
    fun create(@RequestBody book: Book): ResponseEntity<Book> {
        val newBook = bookService.create(book)
        return ResponseEntity.created(URI.create("/books/${newBook.id}")).body(newBook)
    }

    @GetMapping
    fun getBooks() = ResponseEntity.ok(bookService.getAll())

    @GetMapping("/{id}")
    fun findBookById(@PathVariable id: UUID) = ResponseEntity.ok(bookService.findById(id))

    @PutMapping("/{id}")
    fun update(@PathVariable id: UUID, @RequestBody book: Book) = ResponseEntity.ok(bookService.update(id, book))

    @DeleteMapping("/{id}")
    fun delete(@PathVariable id: UUID): ResponseEntity<HttpStatus> {
        bookService.delete(id)
        return ResponseEntity.noContent().build()
    }
}

The Service:

@Service
class BookService(private val bookRepository: BookRepository) {

    fun create(book: Book): Book = bookRepository.save(book)

    fun getAll(): List<Book> = bookRepository.findAll()

    fun findById(id: UUID): Book = findBook(id)

    fun update(id: UUID, book: Book): Book = bookRepository.save(with(book) {
        findBook(id).copy(title = title, author = author)
    })

    fun delete(id: UUID) = bookRepository.delete(findBook(id))

    private fun findBook(id: UUID): Book = bookRepository.findById(id).orElseThrow { BookNotFoundException(id) }
}

The Repository:

@Repository
interface BookRepository : JpaRepository<Book, UUID>

The exception package is not important, but you can check it out in my GitHub Repo.

Until now, It’s a simple CRUD. Nothing special here. Now we can start to add and configure Testcontainers.

Adding Testcontainers dependency

Simply add this Testcontainers dependency at the end of your build.gradle.kts dependencies.

testImplementation("org.testcontainers:testcontainers:1.16.0")

Configuring the base integration test

In order to make all integration tests have the same configuration, we have to create a base integration test, so our tests will inherit the base behavior by extending it.

The first step is to create the BaseIntegrationTest class, that will instantiate the tests in a random port.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class BaseIntegrationTest {

}

Now, we can add a cleanDB method that will truncate all our tables. That’s the method we are going to use before each of our tests, to make sure we have a brand new empty database.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class BaseIntegrationTest {

    @Autowired
    private val jdbcTemplate: JdbcTemplate? = null

    @Transactional
    protected fun cleanDB() {
        val tablesToTruncate = listOf("book").joinToString()
        val sql = """
            TRUNCATE TABLE $tablesToTruncate CASCADE
        """.trimIndent()
        jdbcTemplate?.execute(sql)
    }
}

I’m using JDBC to truncate the tables, but you can use any method you want.

Testcontainers initializer

To initialize the containers, we have to create an initializer for the tests, so let’s create the BaseITInitializer, that will implement the ApplicationContextInitializer.

class BaseITInitializer : ApplicationContextInitializer<ConfigurableApplicationContext> {  

    override fun initialize(applicationContext: ConfigurableApplicationContext) {  
        TODO("Not yet implemented")  
    }  
}

The next step is to configure a docker-compose file to initialize our containers. At the root of the project, let’s create a docker directory with a docker-compose.yml inside.

version: '3'  

services:  
  postgres:  
    image: postgres:9.6  
  environment:  
      POSGRES_USER: testcontainers  
  POSTGRES_PASSWORD: 1234  
  POSTGRES_DB: testcontainers_demo  
  ports:  
      - "5432:5432"

Now we can start to create our containers using docker-compose as parameter.

class BaseITInitializer : ApplicationContextInitializer<ConfigurableApplicationContext> {  

    companion object {
        class KDockerComposeContainer(file: File) : DockerComposeContainer<KDockerComposeContainer>(file)

        class Container(
            val serviceName: String,
            val port: Int
        )

        private val POSTGRES = Container("postgres_1", 5432)

        private val COMPOSE_CONTAINER: KDockerComposeContainer by lazy {
            KDockerComposeContainer(File("docker/docker-compose.yml"))
                .withExposedService(POSTGRES.serviceName, POSTGRES.port)
                .withLocalCompose(true)
        }
    } 

    override fun initialize(applicationContext: ConfigurableApplicationContext) {  
        TODO("Not yet implemented")  
    }
}

It’s time to implement the initialize method, by initializing our compose container and defining the environment variables.

class BaseITInitializer : ApplicationContextInitializer<ConfigurableApplicationContext> {  

    companion object {
        class KDockerComposeContainer(file: File) : DockerComposeContainer<KDockerComposeContainer>(file)

        class Container(
            val serviceName: String,
            val port: Int
        )

        private val POSTGRES = Container("postgres_1", 5432)

        private val COMPOSE_CONTAINER: KDockerComposeContainer by lazy {
            KDockerComposeContainer(File("docker/docker-compose.yml"))
                .withExposedService(POSTGRES.serviceName, POSTGRES.port)
                .withLocalCompose(true)
        }
    }

    override fun initialize(applicationContext: ConfigurableApplicationContext) {
        COMPOSE_CONTAINER.start()

        val jdbcURL = "jdbc:postgresql://${getContainerUrl(POSTGRES)}:${POSTGRES.port}/testcontainers_demo"

        TestPropertyValues.of(
            "spring.datasource.url=$jdbcURL"
        ).applyTo(applicationContext.environment)
    }

    private fun getContainerUrl(container: Container) =
        COMPOSE_CONTAINER.getServiceHost(container.serviceName, container.port)
}

To finish the configuration, we just need to add our initializer at the base test.

@ContextConfiguration(initializers = [BaseITInitializer::class])  
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)  
class BaseIntegrationTest {  

  @Autowired  
  private val jdbcTemplate: JdbcTemplate? = null  

  @Transactional  
  protected fun cleanDB() {  
        val tablesToTruncate = listOf("book").joinToString()  
        val sql = """  
            TRUNCATE TABLE $tablesToTruncate CASCADE 
        """.trimIndent()  
        jdbcTemplate?.execute(sql)  
    }  
}

Implementing the Integration Tests

Since we have the base test, we can start to implement the BookIntegrationTest, by extending the base, cleaning the database before each test, and initializing the mockMvc with context.

class BookIntegrationTest : BaseIntegrationTest() {  

    private lateinit var mockMvc: MockMvc  

    @Autowired  
    private lateinit var context: WebApplicationContext  

    @BeforeEach  fun setup() {  
        cleanDB()  
        mockMvc = MockMvcBuilders.webAppContextSetup(context).build()  
    }  
}

To test the listing books endpoint, first, we have to populate our empty database with some books. To do that, let’s create a seed.

class BookSeeds(private val bookRepository: BookRepository) {  

    fun insertMultipleBooks() {  
        bookRepository.saveAll(  
            listOf(  
                Book(  
                    id = UUID.fromString("1e162d4e-66a4-47ba-85bd-9019dedc30a2"),  
                    title = "Title 1",  
                    author = "Author 1"  
                ),  
                Book(  
                    id = UUID.fromString("1b5ba5f8-b99f-4421-8f0c-7048263a10bb"),  
                    title = "Title 2",  
                    author = "Author 2"  
                ),  
                Book(  
                    id = UUID.fromString("dd11fc32-bcaa-414d-9c8c-89a50bd05acc"),  
                    title = "Title 3",  
                    author = "Author 3"  
                )  
            )  
        )  
    }  
}

I’m using Spring Data JPA to populate the table with books, but you can use other methods, like JDBC Template, Flyway, Liquibase, etc…

Now, we have to generate the books after cleaning the database.

class BookIntegrationTest : BaseIntegrationTest() {  

    private lateinit var mockMvc: MockMvc  

    @Autowired  
    private lateinit var context: WebApplicationContext  

    @Autowired  
    private lateinit var bookRepository: BookRepository  

    @BeforeEach  
    fun setup() {  
        cleanDB()  
        BookSeeds(bookRepository).insertMultipleBooks()  
        mockMvc = MockMvcBuilders.webAppContextSetup(context).build()
    }
}

Add books listing test, and test it! (Remember to have docker up and running, and nothing can be running on port 5432)

class BookIntegrationTest : BaseIntegrationTest() {  

    private lateinit var mockMvc: MockMvc  

    @Autowired  
    private lateinit var context: WebApplicationContext  

    @Autowired  
    private lateinit var bookRepository: BookRepository  

    @BeforeEach  
    fun setup() {  
        cleanDB()  
        BookSeeds(bookRepository).insertMultipleBooks()  
        mockMvc = MockMvcBuilders.webAppContextSetup(context).build()  
    }  

    @Test  
    fun `should return all books`() {  
        val request = MockMvcRequestBuilders.get("/books")  
        val response = mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isOk)  
        val jsonArray = JSONArray(response.andReturn().response.contentAsString)  

        // Http verification
        assertEquals(jsonArray.length(), 3)  
    }  
}

Add more tests…

class BookIntegrationTest : BaseIntegrationTest() {

    private lateinit var mockMvc: MockMvc

    @Autowired
    private lateinit var context: WebApplicationContext

    @Autowired
    private lateinit var bookRepository: BookRepository

    private val bookId = UUID.fromString("1e162d4e-66a4-47ba-85bd-9019dedc30a2")

    @BeforeEach
    fun setup() {
        cleanDB()
        BookSeeds(bookRepository).insertMultipleBooks()
        mockMvc = MockMvcBuilders.webAppContextSetup(context).build()
    }

    @Test
    fun `should return all books`() {
        val request = MockMvcRequestBuilders.get("/books")
        val response = mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isOk)
        val jsonArray = JSONArray(response.andReturn().response.contentAsString)

        // Http verification
        assertEquals(jsonArray.length(), 3)
    }

    @Test
    fun `should return book by id`() {
        val request = MockMvcRequestBuilders.get("/books/$bookId")
        val response = mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isOk)
        val jsonObject = JSONObject(response.andReturn().response.contentAsString)

        // Http verification
        assertEquals(jsonObject.get("id"), bookId.toString())
        assertEquals(jsonObject.get("title"), "Title 1")
        assertEquals(jsonObject.get("author"), "Author 1")
    }

    @Test
    fun `should create book`() {
        val body = """
            {
                "title": "A title",
                "author": "An author"
            }
        """.trimIndent()

        val request = MockMvcRequestBuilders
            .post("/books")
            .contentType(MediaType.APPLICATION_JSON)
            .content(body)

        val response = mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isCreated)
        val jsonObject = JSONObject(response.andReturn().response.contentAsString)

        // Http json verification
        assertNotNull(jsonObject.get("id"))
        assertEquals(jsonObject.get("title"), "A title")
        assertEquals(jsonObject.get("author"), "An author")

        // Database verification
        val book = bookRepository.findById(UUID.fromString(jsonObject.get("id").toString()))
        assertEquals(book.get().title, "A title")
        assertEquals(book.get().author, "An author")
    }

    @Test
    fun `should update book`() {
        val body = """
            {
                "title": "Updated title",
                "author": "Updated author"
            }
        """.trimIndent()

        val request = MockMvcRequestBuilders
            .put("/books/$bookId")
            .contentType(MediaType.APPLICATION_JSON)
            .content(body)

        val response = mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isOk)
        val jsonObject = JSONObject(response.andReturn().response.contentAsString)

        assertEquals(jsonObject.get("id"), bookId.toString())
        assertEquals(jsonObject.get("title"), "Updated title")
        assertEquals(jsonObject.get("author"), "Updated author")

        // Database verification
        val book = bookRepository.findById(UUID.fromString(jsonObject.get("id").toString()))
        assertEquals(book.get().title, "Updated title")
        assertEquals(book.get().author, "Updated author")
    }

    @Test
    fun `should delete book by id`() {
        val request = MockMvcRequestBuilders.delete("/books/$bookId")
        mockMvc.perform(request).andExpect(MockMvcResultMatchers.status().isNoContent)

        // Database verification
        val bookExists = bookRepository.existsById(bookId)
        assertFalse(bookExists)
    }
}

It’s working!

This is our finished structure:

The full project can be found on my Github.

Conclusion

Testcontainers can provide a whole environment to implement integration tests. In this post, I covered only database proposes, but you can actually add more infrastructure to cover, like caching, mocking external requests with Mock Server, etc.

There are a lot of ways to implement integration tests, and Testcontainers is a really good choice, so give it a try!

Thank you for reading, and don’t stop learning!

We are hiring new talents. Do you want to work with us? become@codeminer42.com