Contract Tests

What is Contract testing

An approach that captures the interactions between two components and serialises them into a contract, which can then be used to verify that both parties adhere to it.

— this quote is looking for its author

Handled Today?

  • Lets do a v2/

  • Not at all aka commit and pray

  • Manual research into client code

  • Hand written Test Doubles

  • Integration tests across services

  • Integration environments

  • End to End tests across the whole system

Integrated tests are a scam — a self-replicating virus that threatens to infect your code base, your project, and your team with endless pain and suffering.

... I use the term integrated test to mean any test whose result (pass or fail) depends on the correctness of the implementation of more than one piece of non-trivial behavior.

October 16, 2010 Integrated Tests Are a Scam
— J.B Rainsberger

pact.io

Enables consumer driven contract testing, providing a mock service and DSL for the consumer project, and interaction playback and verification for the service provider project.

— the pact website

Contracts should be about catching …​

  • bugs in the consumer

  • misunderstanding from the consumer about end-points or payload

  • breaking changes by the provider on end-points or payload

Pact scenarios should not dig into the business logic of the Provider.

What can be improved

  • Feature parity between implementations (v2, v3, v4)

  • Documentation (pact-foundation consolidation)

  • Pact-Jvm

    • is implemented in Java/Scala/Kotlin/Groovy

    • configuration via system properties

    • documentation spread around in Readme.md files

What to gain by using pact (1)

  • No more integration tests !?

  • Generated Test Doubles

  • Knowing your clients

    • what properties are they using

    • who to contact in case of changes

    • you get people talking and clean up wrong assumptions

What to gain by using pact (2)

  • Pact-Broker as …​

    • contract repository

    • source of truth for pre deployment checks

    • displaying service dependencies

I’m intrigued, what next

Where to get help

That’s it!
Any Questions?

Beyond Consumer Driven Contracts

Session offer from Nicole for April

Pact: Matchers

Request matching (when does the mock provider return the expected values) * no unexpected values in the json body or query string * unexpected headers are allowed

Response matching (during verification on the provider side) * make sure the types are matching (Array, String, boolean, Fixnum, …​) * unexpected values in JSON are ignored * it is not possible to verify that something is not there.

Pact: Terminology

Consumer

A component that initiates a HTTP request to another component (the service Provider).

Provider

A server that responds to an HTTP request from another component (the service Consumer).

Interaction

One invocation of the Provider by the Consumer including send arguments and expected response.

Contract

A file containing the JSON serialised interactions (requests and responses) that were defined in the Consumer tests. (aka the pact file)

Pact: Example

"Given …​ a request for …​ will return …​"

Given an alligator with the name Mary exists *
Upon receiving a request to retrieve an alligator by name ** from Some Consumer
With {"method" : "get", "path" : "/alligators/Mary" }
Some Provider will respond with { "status" : 200, ...}

* This is the provider state
** This is the request description

Pact: Verification

would add a caution to make sure you are not affecting behavior with your mocks and stubs. This is about running the service as is

Pact: How to keep the pact file in sync?

Instead of having to build a test environment running with two servers and databases to test that our interaction between our source system and our event API works, we now have two fast tests that achieve the same result. As long as we maintain the pact file between the two systems, we can be reasonably sure that our integration is working correctly.

There is a number of ways of maintaining the pact file. We can get our consumer build to publish the pact file to the CI server as a build artifact. The pact verification task supports loading the pact file from a URL. That way we can get it to always use the pact file from the latest successful build.

There is also a Pact Broker project that provides a repository of pact files. It allows you to be able to verify your providers against pact files from different versions of your consumer.

Pact Broker

  • keep pacts in sync

  • map the relationships between your services

  • track compatibility across different versions

  • free hosted pact brokers

Sources

Pact: Supported languages

Pact: Tools

Pact: What changed between versions

v1

v2

Auf 3: provider state → provider states message queues

If you are writing tests on the Consumer side to a different language on the Provider side, you must ensure you use a common Pact Specification between them or you will be unable to validate.

Pact: Where to use it

  • If you control code for the consumer and the provider

  • public apis are not suited, better of with Swagger documentation

Pact: What can not be ensured with this approach

  • Firewall rules and network connectivity