Pact.NET: Bi-Directional CT & DevOps Pipelines

Table of contents

Overview

Here we add examples of how different parts of the DevOps YAML-based provider and consumer pipelines for a Pact.NET-based bi-directional contract testing solution could be implemented.

This implementation example assumes a .NET-based provider and consumer. But the idea can pretty easily be transferred to non .NET-based providers and or consumers. The reason we have limited ourselves here to .NET is that we want to use dotnet swagger on the provider side to generate the OpenAPI file from code automatically and because we want to use Pact.NET on the consumer side to generate the pact file.

This is not a tutorial but just some pieces of information and ideas on how you could implement parts of your pipeline.

Provider Pipeline

In our solution the provider pipeline is responsible to perform the following tasks in this order:

Automatically generate a swagger.json (OpenAPI) file. This file should contain the definition of the provider API and should be generated automatically from the API code.
Automatically create test cases generated based on swagger.json (OpenAPI) and run the tests against the real provider API. The results of these tests are stored in a file e.g. results.txt.
Then both files are published to the Pactflow server. The swagger.json file serves as the contract which is then checked against the consumer side. The results.txt file serves as evidence that we have tested the provider and are confident that it fulfills its own definition.
Then the Pact can-i-deploy command is executed
Then the Pact record deployment command is executed.

Generating the swagger.json file

The swagger.json file could be generated with the dotnet swagger command:

dotnet swagger tofile --output swagger.json providerApp.API/bin/Release/net6.0/providerApp.API.dll v1

This command will generate the swagger.json file which will be published to the Pactflow server in a later step.

More info about how to generate the swagger.json file can be found here: https://www.iteacorner.com/swashbuckle-and-asp-net-core/

Start your Application in the pipeline

The following command shows an example of how we could run our provider in a container in the pipeline as a background process, so the pipeline is not blocked by the running provider app and we can proceed to start the test tool.

...
- pwsh: |     
    ...
    $job = Start-Job { docker run -e IS_API_TEST=true --rm -p 80:80 --name providerAppcontainer $(docker images --filter "label=proivderApp=true" -q) }
...

Run your test tool in parallel to your application in the pipeline

Now that our provider application runs in the background in a container. We can start the testing tool (e.g. dredd, which is a tool capable of creating automatic tests based on the swagger.json of the provider API) in the pipeline as a next step:

...
- pwsh: |
  ...  
    dredd swagger.json http://localhost:80  -h "Authorization:Bearer $someToken" --loglevel debug | Tee-Object -FilePath "results.txt"
...

The dredd test tool should be able to automatically generate test cases based on the swagger.json we provide to it. You can also use other tools like Schemathesis instead which are also capable to automatically generate test cases based on swagger.json definitions and run tests against the API.

Publishing results evidence and swagger.json

The following command is an example how the test results evidence and swagger.json can be published to the Pactflow server:

- pwsh: |
  ...
    $OpenAPIFilePath = "swagger.json"
    $ResultsFilePath = "results.txt"
    $ExitCode = 0 or better the actual result of the dredd test!
    $TestTool= "dredd"
    $Pacticipant = "providerApp"
    $ResultsContentType = "text/plain"
    
    # NOTE!!! Make sure that your URL does not end with slash, because the pactflow publish-provider-contract command seems to have a problem with that!
    $PactBrokerBaseUrl = "https://your-company.pactflow.io" 
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...)
 
    docker run --rm -v ${PWD}:${PWD} -w ${PWD} pactfoundation/pact-cli:latest pactflow publish-provider-contract ${pwd}/$OpenAPIFilePath `
        --broker-base-url $PactBrokerBaseUrl `
        --broker-token $PactBrokerBaseToken  `
        --provider $Pacticipant `
        --provider-app-version $(Build.SourceVersion) `
        --branch $(Build.SourceBranchName) `
        --content-type application/yaml `
        --verification-exit-code=$ExitCode `
        --verification-results ${pwd}/$ResultsFilePath `
        --verification-results-content-type $ResultsContentType `
        --verifier $TestTool `
        --verbose
  ...

can-i-deploy command

This is an example how the can-i-deploy command might look like:

- pwsh:  |
  ...
    $Pacticipant = "providerApp" 
    $PactBrokerBaseUrl = "https://your-company.pactflow.io" 
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...)

    docker run --rm -v ${PWD}:${PWD} -w ${PWD} pactfoundation/pact-cli:latest pact-broker can-i-deploy `
      --broker-base-url $PactBrokerBaseUrl `
      --broker-token $PactBrokerBaseToken `
      --pacticipant $Pacticipant `
      --version $(Build.SourceVersion) `
      --to-environment DEV
  ...

Record deployment

This is an example how the record deployment command might look like:

- pwsh:  |
  ...
    $Pacticipant = "providerApp" 
    $PactBrokerBaseUrl = "https://your-company.pactflow.io" 
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...

    docker run --rm -v ${PWD}:${PWD} -w ${PWD} pactfoundation/pact-cli:latest pact-broker record_deployment `
       --broker-base-url $PactBrokerBaseUrl `
       --broker-token $PactBrokerBaseToken `
       --pacticipant $Pacticipant `
       --version $(Build.SourceVersion) `
       --to-environment DEV
  ...

Consumer Pipeline

For the consumer side we use Pact.NET to generate a pact file in an unit test.
We don’t want to go in details here how to implement such a unit test because there are greate example on the Pact site itself:

https://docs.pactflow.io/docs/examples/bi-directional/consumer/dotnet/

Bi-Directional Pact allows different ways to generate a pact file, using pact or other tools. The tested solution behind this implementation was using Pact.NET to generate the consumer pact.
In our example somewhere in the build pipeline the Pact.NET unit test is executed with a command like this:

dotnet test ./cusmerApp.PactUnitTest/cusmerApp.PactUnitTest.csproj --configuration Release --logger "trx;LogFileName=pact_test_results.xml" --results-directory /results

But this is just an example, due to flexibility of Bi-Direct pact, the consumer pact file could also be generated by other means. Here we want to focus on the pipeline implementation and therefore we just assume that the pact file is somehow created by Pact.NET or by any other means.

Publishing the pact file

The consumer pact file could be generated in the pacts folder and the following command could publish it to the Pactflow server:

- pwsh: |
  ...
    $PactBrokerBaseUrl = "https://your-company.pactflow.io"
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...)

    docker pull pactfoundation/pact-cli:latest
    docker run --rm -v ${PWD}:${PWD} pactfoundation/pact-cli publish ${PWD}/pacts -b $PactBrokerBaseToken -k $PactBrokerBaseToken --consumer-app-version $(Build.SourceVersion) --auto-detect-version-properties --branch $(Build.SourceBranchName) --build-url $(Build.BuildUri)
  ...

can-i-deploy command

This is an example how the can-i-deploy command might look like:

- pwsh:  |
  ...
    $Pacticipant = "consumerApp" 
    $PactBrokerBaseUrl = "https://your-company.pactflow.io"
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...)

    docker run --rm -v ${PWD}:${PWD} -w ${PWD} pactfoundation/pact-cli:latest pact-broker can-i-deploy `
	--broker-base-url $PactBrokerBaseUrl `
	--broker-token $PactBrokerBaseToken `
	--pacticipant $Pacticipant `
	--version $(Build.SourceVersion) `
	--to-environment DEV
  ...

Record deployment

This is an example how the record deployment command might look like:

- pwsh:  |
  ...
    $Pacticipant = "consumerApp" 
    $PactBrokerBaseUrl = "https://your-company.pactflow.io"
    $PactBrokerBaseToken = ... (hard coded token or better passed from an environment variable ...)

    docker run --rm -v ${PWD}:${PWD} -w ${PWD} pactfoundation/pact-cli:latest pact-broker record_deployment `
        --broker-base-url $PactBrokerBaseUrl `
        --broker-token $PactBrokerBaseToken `
        --pacticipant $Pacticipant `
        --version $(Build.SourceVersion) `
        --environment DEV
  ...

Debugging

Because running the pipeline is a time-consuming task, it can be useful to have a local script to test the commands in the pipeline locally. Here is an example of a bash script to execute pipeline commands locally, in this case, the pact publishing command:

#!/bin/bash

BuildSourceVersion=1000
BuildSourceBranchName='SomeBranch'
BuildBuildUri='http://SomeDummyUrl'

PactBrokerBaseUrl="https://your-company.pactflow.io"
PactBrokerBaseToken=... (hard coded token or better passed from an environment variable ...)

docker pull pactfoundation/pact-cli:latest

docker run --rm -v ${PWD}:${PWD} pactfoundation/pact-cli publish ${PWD}/pacts -b "$PactBrokerBaseUrl" -k "$PactBrokerBaseToken" --consumer-app-version "$BuildSourceVersion" --auto-detect-version-properties --branch "$BuildSourceBranchName" --build-url "$BuildBuildUri"

admin

Uncategorized