Contract test instead of integration tests
In a previous article published by my colleagues about how to use Pact and MockK, they well explained how to implement contract testing.
Still, I’d like to go in deep on why in some cases, contract tests are the best solution to verify the proper behavior when,
in a microservice’s architecture, two or more components need to exchange a message.
Why don’t we use integration tests also to verify the correct integration between two microservices? We can run these
tests in our QA environment and be sure that this integration works.
For some reason, in lastminute.com group, we used to call Integration Test also the tests
which made actual calls between two microservices, and interact with databases and all the other external components,
even if they were supposed to be more likely system tests. So when in this article I will mention IT tests, I will refer
to those.
In the past, we had tons of IT tests, but unfortunately, they were weak and failure-prone for reasons different from
what we aimed to verify.
For example, they used to fail because the called service was unreachable in the QA environment at that moment but was
perfectly working in the production environment. Another reason for which such tests were failing was because they were
also integrating with our QA database, and to give an answer during the test phase, we were reading some rows, but someone
changed those rows manually for testing reasons. The result was always the same: these tests were ignored, and none
of these tests covered the integration between our beloved microservices.
Here we can see a simplified version of such a kind of test:
public class CallServiceTestIT{
@Test
@Throws(InterruptedException.class)
public void testCall(){
String id = UUID.randomUUID().toString();
Client client = new Client();
ServiceResponse response = client.callWithId(id);
assertThat(response.getStatusCode(),is(200));
assertThat(response.getId(),is(id));
assertThat(response.getLanguage(),is("en"));
assertThat(response.getPrice(),is(205.57d));
}
}In the example above, we are creating the client used to make the call, the unique identifier, the id for what we aim to
retrieve, and the assertions for the result. Indeed, this test is easy to write and read, but unfortunately, we cannot
say it’s easy to maintain: since we are dealing with a real rest call to another service, many things can go wrong.
My colleagues have provided an obvious and complete example of a contract test in the article linked above: I
suggest to have a look at it, because it explains a complete example contract test with Pact.
Using contract tests, we verify that when a microservice sends a message of any kind, the answer will be the one we
expect, without needing an actual real response from the target microservice. Also, one of the reasons we write
tests is to receive feedback: the shorter the feedback cycle, the faster the development process.
There are two main ways to implement contract tests: consumer-driven and producer-driven.
In this article, I will refer to consumer-driven ones, but the same considerations are still valid for any kind of
contract tests.
Impact on the feedback cycle and development process
As mentioned before, when we used to implement integration tests to verify the message exchange between two microservices, they failed for multiple reasons. This weakness leads us to ignore those tests to avoid blocking the development process. The result was that they were left ignored for months, and when something changed on the other side, both the CI pipelines were green: usually, we realized there was an error in the contract when something in production failed. We need time to investigate the reasons for the failure, involve other teams and ask for a rollback, or fix the issue as fast as possible.
For example, as we can see in the message below, our colleague Roberto asked for help to fix a test failure in QA.
Other times, the root cause of the test failure was located in some data stored on a database that someone or some
other application changed. In these cases, debugging was even more expensive in terms of effort and time.
Consequently, the time to market for what we need to release expanded.
Since none of us liked to take action to fix tests for almost every release, these tests were marked as ignored, with
the result that nothing covered the message exchange.
Since in a microservice architecture the communication part is an important element of every microservice, we need
these steps covered by tests, because, instead of having communication inside the monolith, in a distributed system we
have remote communication who works through interconnection of systems, who must talk through the same language, or in
our context, must have the same contract.
There needs to be more than testing the contract to verify the proper communication of the microservices. Still, since this kind of test is localized to the contract, it will fail if it has not been respected or has been changed in one of the two parts (caller or called service), and the debug part requires only the time to check the changes on the contract itself.
Impact on the feedback cycle and the lead time
Let’s now consider the execution of a common Spring application that runs IT tests. When they are executed, they
need to start the entire application and then run the tests.
To do so, it needs to create all the configuration objects, connect to databases, message brokers, and so on:
considering a typical application we have in lastminute.com group, the startup time
itself can be around a minute, and, it still has to run the tests.
Also, to execute an integration test with a simple call, we need to start our service, call the target and wait for the
response. When each one is connected to many others, the testing execution time will increase rather quickly, and so
will our feedback cycle.
Instead, to execute the Pact tests, we don’t need the application to be up and run (locally or in the CI tool), but we need to contact the Pact broker: the execution time for a few tests is less than a second! Consequently, the test execution time is shorter. We can react faster and reduce our development time and, of course, the cycle time.
Contract tests are not enough
With contract tests, we verified that our microservices “speak the same language”, but it is not enough to prove the whole communication part. All the communication steps consist of many components, like, in the Rest sync communication, the call logic with its faults (since we are using infrastructure and network), and the controller and its logic. On both sides, we need to verify the behaviors. Still, since we excluded any possible fault due to the contract, we can (we must!) test them in isolation, with a low execution time and, not less significantly, easier to understand and debug.
