Property-based Testing for API Workflows

25 Jun, 2026

TLDR Cuerdo turns API specifications into hundreds of E2E tests. Based on open standards, you can write an Arazzo document and Cuerdo finds edge cases automatically instead of manually inspecting OpenAPI schemas and writing potential edge cases.

The problem

API testing is still not a solved problem. For exploration and manual API workflows testing, teams usually rely on tools such as Postman, Insomnia, and Bruno. These tools make it easy to interactively run requests, inspect responses, and combine multiple request/responses into a single workflow. However, they’re treated as documentation or base reference, and often not integrated into CI pipelines. As a result, the “less often executed” workflows tend to go out of sync, no longer reflecting the API contract/behavior.

For automated API testing, Playwright, Cypress, and custom frameworks are more commonly used. These are much easier to integrate into CI pipelines and provide better confidence, but since they’re often maintained by different teams, over time they might drift and not exercise newer options, filters, etc. from the APIs.

Even when managing to automate API workflow testing, and integrating them in CI pipelines, a limitation remains: workflows run against a single input o a small set of known inputs. This is enough for regression testing, making sure nothing breaks, but might not catch bugs in edge cases. Unless a QA or developer considers every edge case, these can go unnoticed for a long time in production until a user triggers an edge case. What happens later is usually a developer fixing the bug and adding the failing case to Playwright/Cypress or as a unit test in the server code.

Usually teams end up maintaining a combination of the three: interactive UI and “documentation” reference in Postman, Playwright for automation and CI, and a few unit tests in the app code when someone finds and fixes a new bug. Keeping everything synchronized requires significant effort and continuous cross-team communication between developers and QA.

What if there was a way to write API workflows that’s easy to understand by developers and QA, easy to automate and include in CI pipelines, and could generate hundreds of test cases automatically?

The solution

The specification for defining workflows exists: the Arazzo spec.

Arazzo is an open specification format from the OpenAPI Initiative, and builds on top of OpenAPI. OpenAPI specifies individual API requests and responses, while Arazzo combines request and responses into workflows. Arazzo is tool agnostic, there is no vendor lock-in and it’s not tied to a specific language. Like OpenAPI, anyone can build tools, linters, runners, visualizers, etc. around the specification.

Arazzo documents are usually in YAML format, which more version-control friendly, easy to understand by developers and QA, and most developers are familiar with YAML because of CI/CD pipelines and Kubernetes manifests. No need to open a specific tool to edit workflows files, Arazzo workflows are regular files that you can keep in the same codebase as the app code.

So, there is a way to describing workflows, but how to generate tests and run them automatically? Meet Cuerdo

Cuerdo is a command-line tool, and Elixir library, that takes an Arazzo document and generates property-based tests for each workflow. Instead of having a fixed set of inputs, Cuerdo generates random data based on the workflow inputs definition, which are in JSON Schema format, and executes the workflow from start to finish, validating at each step that:

Request and response match the OpenAPI specification
Step an workflow success criteria match on every request and response

When Cuerdo encounters a failing case, it stores the entire request/respose logs in a HAR-like format, so developers can easily identify and reproduce the failure.

You can find more information in documentation or check the repository and give it a try.

Example

Just to show what an Arazzo workflow looks like, here is a basic user creation + retrieving id workflow, where the user authenticates to access their profile. The full Arazzo document should contain extra fields but this example shows only the workflows attribute for conciseness

workflows:
  - workflowId: userCreation
    inputs:
      # JSON Schema representing the inputs object
      type: object
      properties:
        email:
          type: string
          format: email
        password:
          type: string
          minLength: 8
          maxLength: 32
      required: ["email", "password"]
      additionalProperties: false
    steps:
      - stepId: createUserStep
        operationId: createUser # "createUser" defined in the OpenAPI document
        requestBody:
          contentType: application/json
          payload:
            email: $inputs.email
            password: $inputs.password
            passwordConfirmation: $inputs.password
        successCriteria:
          - condition: $statusCode == 201
          - type: jsonpath
            condition: $.meta.token
            context: $response.body
        outputs:
          token: $response.body#/meta.token
          userId: $response.body#/id
      - stepId: getUserProfile
        operationId: getUser
        parameters:
          - name: Authorization
            in: header
            value: Bearer $steps.createUserStep.outputs.token
          - name: userId
            in: path
            value: $steps.createUserStep.outputs.userId
        successCriteria:
          - condition: $statusCode == 200
          - condition: $response.body#/email == $inputs.email

The workflow creates a user with email and password, checks that the response has status 201 CREATED with a token and the userId in the body. It then uses the token for basic authentication to get the user profile, probably at /users/{userId} given that it injects userId in the path parameters, and expects that the queried user email matches the input.

Cuerdo can generate test cases from the inputs schema, without additional code or specifications. It understands "format": "email" from the JSON schema. For example, it generated the following 10 valid emails when passing the Arazzo document:

["[email protected]", "[email protected]", "[email protected]", "[email protected]",
 "[email protected]", "[email protected]", "[email protected]",
 "[email protected]",
 "zRo&s&@K.UGBt.Y.S.Z.q.XRiF--2.B1-l8--v-7j.mT",
 "&/X'qTe@s.X-2---L.JlqtowD-CZ06.T8h.J.dPh---7ifW.f.u.V3-r-k-2b.wInnErS"]

There is no need to manually think of edge cases and test them manually one at a time. Cuerdo automatically generates and executes workflow variations from a single Arazzo document. The workflow definition is now documentation and an executable test, reducing the maintenance burden of multiple tools and environments :)