API Design and Governance: Versioning, Idempotency, and Contract Tests

A contract is more than a schema

A schema describes fields and types. A working API contract also describes behavior: whether a missing field has a default, whether an operation can be retried, how errors are represented, what authorization is required, and how long a client can rely on the current behavior. Two systems can pass schema validation and still disagree about whether a payment is final, whether an empty list differs from a missing list, or whether a retry creates a second order.

That distinction matters most when producers and consumers are deployed independently. A service may have dozens of clients: a mobile application released through app stores, an internal batch job, a partner integration, and services owned by other teams. They do not all upgrade together. API governance is the set of engineering rules that lets those clients move at different speeds without making every deployment a coordinated event.

Start by writing down the observable promises. For a create-order operation, that might include accepted request fields, authorization scope, response status codes, whether the response contains a stable order identifier, and how the client recovers if it never receives the response. These statements give implementation, review, testing, and operations teams the same boundary to protect.

Choose REST or gRPC for the interaction, not the fashion

REST and gRPC offer different constraints and strengths. REST over HTTP is often a practical fit for public APIs, browser-based clients, and integrations where ordinary HTTP tooling and readable payloads are valuable. Resource-oriented paths and standard HTTP semantics make many operations straightforward to inspect with a command-line client or gateway. But REST is not automatically consistent: teams still need shared rules for pagination, error bodies, filtering, and conditional requests.

gRPC uses Protocol Buffers and generated client and server code. It is a strong choice for service-to-service calls where typed interfaces, efficient serialization, streaming, and code generation fit the environment. Its stronger schema tooling does not eliminate compatibility risk; it makes some risks easier to detect. A field can be syntactically valid and still be misunderstood by an older client, and a semantic change can break consumers even if the protobuf definition remains unchanged.

Make the choice against the boundary in front of you. Consider the clients’ languages and deployment cadence, network intermediaries, streaming needs, browser access, debugging workflows, and whether a gateway is required. A gRPC service exposed through a JSON transcoding layer has two representations to govern, not one. Likewise, an HTTP API consumed only by internal services still needs stable semantics and a clear policy for incompatible changes.

Whichever transport you choose, define an error model deliberately. Clients need to distinguish validation failures from authentication failures, throttling, transient dependency failures, and unknown server errors. For HTTP, use status codes consistently and provide a machine-readable error body; RFC 9457 defines the Problem Details format for HTTP APIs. For gRPC, use status codes and structured error details where appropriate. Do not make clients parse human-oriented error strings to decide whether to retry.

Compatibility is about meaning as well as shape

A change is backward compatible only if existing clients continue to work as intended. Adding an optional response field is generally safer than removing one, because clients that ignore unknown fields can continue. Adding a required request field is usually breaking: older clients will omit it. Renaming a field, changing its units, narrowing an accepted value range, or changing an error from retryable to terminal can also break clients without altering the endpoint.

Compatibility rules should be explicit for requests and responses. For requests, a server can often accept a new optional field while continuing to support the old form. For responses, adding fields is safe only if clients are expected to tolerate fields they do not recognize. Some generated clients or strict decoders reject unknown properties, so verify actual consumer behavior rather than relying on a theoretical standard.

Enums deserve special attention. A server that adds a new enum value may break an older client whose code assumes the set is exhaustive. In a protobuf API, an older client may receive an unknown enum numeric value; its behavior depends on the language and generated runtime. In JSON APIs, strict deserialization can fail outright. Either design clients to handle unknown values, or treat enum expansion as a compatibility event requiring coordinated controls.

Field names can conceal semantic drift. If amount changes from dollars to cents, the schema may still say “integer,” but the contract has changed radically. Prefer explicit units in names or documentation, define currency and rounding rules, and introduce a new field or version when the original meaning cannot be preserved. A stable field with unstable meaning is not a stable contract.

For Protocol Buffers, deleted field numbers must never be reused. Reserve both the field number and, where useful, the field name in the message definition. Reusing a number can cause old data or clients to interpret a new field as the deleted one. The protobuf documentation’s guidance to reserve deleted identifiers is a small rule with a large blast radius: it protects compatibility across generated code, stored messages, and long-lived deployments.

Write compatibility policy as a reviewable checklist, not an informal expectation. Specify which changes are allowed within a version, how unknown fields and enum values are handled, what constitutes a breaking semantic change, and what evidence is required before removal. Policy that is too strict can freeze useful evolution; policy that is too vague shifts risk onto each consumer team.

Version only when compatibility cannot carry the change

There is no single versioning scheme that fits every API. A path such as /v1/orders makes the major version visible and can simplify routing and client configuration. A version in a media type or header keeps the resource path stable but requires more careful tooling and inspection. For gRPC, package and service names are part of the interface; teams commonly preserve compatible protobuf evolution and introduce a new service or package when a genuinely incompatible contract is needed.

Version numbers do not make a breaking change safe. They create a place for the new contract to live. The cost begins when the old version must remain operational: separate code paths, documentation, monitoring, tests, incident response, and a migration plan. Before creating a version, ask whether additive evolution, a new endpoint, or a new operation can solve the problem without duplicating the whole surface.

When a major version is warranted, publish the reason, migration steps, and support window. Set a deprecation date and a removal date, then make those dates visible in documentation and release notes. Where applicable, RFC 8594 defines the HTTP Sunset response header to communicate when a resource is expected to become unavailable. A header is not a substitute for direct communication with known high-volume consumers, but it can help clients and tooling detect the transition.

Measure usage before removal. Logs or API gateway metrics can identify which client credentials, routes, or versions remain active, provided that collection respects privacy and data-retention requirements. An endpoint with low traffic may still serve a critical month-end process. Pair usage data with named ownership and a migration contact; a graph alone cannot tell you whether a remaining call is safe to retire.

For internal APIs, do not assume that “internal” means “all clients can update today.” A rolling deployment may leave old and new service instances running simultaneously. A safe transition often follows an expand-and-contract sequence: first deploy consumers that understand both forms, then deploy producers that emit the new form, migrate remaining traffic, and only then remove the old form. The exact order depends on which side reads and writes each representation.

Idempotency keys make ambiguous retries manageable

A client timeout does not tell the client whether a write failed. The server may have committed the transaction and lost the response on the way back. Retrying a non-idempotent POST can then create a second order, charge, or job. An idempotency key gives the server a way to recognize that the retry represents the same logical operation.

The client generates a high-entropy key for one intended action and sends it with the request. The server scopes that key to the authenticated caller and operation, then records the request’s outcome. If the same caller repeats the same request with the same key, the server returns the original result instead of applying the write again. A key should not be treated as a general-purpose cache key or as proof of identity; authentication and authorization still apply on every request.

A robust implementation stores enough information to detect accidental key reuse with a different payload. One common design records the key, caller or tenant identifier, operation name, canonical request hash, processing state, response status, response body or durable result reference, and expiration time. The unique constraint should cover the key’s scope, for example (tenant_id, operation, idempotency_key). Without a database-enforced uniqueness boundary, two concurrent retries can both pass an application-level “does this key exist?” check and execute twice.

Consider a payment endpoint. The first request with key pay-7f2… inserts an in-progress record and begins the transaction. If the process commits the charge but crashes before returning a response, the retry finds the record and returns the stored result. If the first request is still running when a retry arrives, the service needs a defined response: wait for the original, return a conflict or in-progress status, or provide a pollable operation resource. Leaving this race unspecified invites client-specific workarounds.

Key lifetime is a business and risk decision. A short retention window saves storage but may allow a late retry to repeat an expensive action. A long window reduces that risk but increases storage and raises questions about key reuse and data retention. Choose a duration based on the maximum realistic retry and reconciliation period, document it, and tell clients not to reuse keys for separate actions. Expiration does not reverse the original operation; it only changes how the server treats later requests.

Also define behavior when the same key arrives with a different payload. Rejecting the request is generally safer than returning a result that the caller did not ask for. Define which errors are persisted: a validation failure may be safe to correct and resubmit with a new key, while a transient failure before any side effect might allow the same key to be retried. Persisting a transient error forever can strand an operation; rerunning an operation after a side effect can duplicate it.

Idempotency is not achieved merely by accepting a header. It depends on atomicity between the business write and the idempotency record. Where one database transaction can cover both, commit them together. If the operation spans systems, such as charging an external processor and writing an order locally, use a durable workflow, an outbox, or a reconciliation process. A local key table cannot guarantee exactly-once effects across a network boundary; the practical goal is controlled retries and a way to detect and repair uncertain outcomes.

Contract tests catch assumptions at the service boundary

Consumer-driven contract testing makes the consumer’s expectations executable. A consumer test records the interactions it relies on—for example, a request to retrieve an order and the response fields needed to render its state. The provider verifies that its implementation still satisfies those expectations before release. Tools such as Pact support this workflow, though the useful part is the disciplined agreement and verification, not the brand of tool.

Keep contracts narrow. A consumer should assert the fields and behavior it actually uses, rather than freezing an entire response document. If it requires orderId and status, it should not fail because an unrelated optional field was added. Overly broad contracts turn harmless evolution into coordination work and encourage teams to bypass the tests.

Provider verification should run against a realistic application boundary, with relevant serialization, routing, and authorization behavior included where feasible. It must also identify which contract revision was verified against which provider build. A green test from last week does not demonstrate that today’s deployment is compatible with today’s consumers.

Contract tests do not replace end-to-end checks. They may not exercise production DNS, TLS, gateway policies, service discovery, deployment configuration, real identity-provider claims, or network timeouts. They also cannot prove that the business meaning of a field matches a consumer’s intent. Use them alongside unit and integration tests, selected end-to-end journeys, and production monitoring. The aim is a layered safety net, not one test category treated as an oracle.

For APIs described with OpenAPI or protobuf, add schema compatibility checks to continuous integration. A compatibility tool can flag removed fields, altered types, or incompatible requiredness before merge. These checks are most useful when the repository defines policy clearly: whether a change must be backward compatible with the previous release, all supported releases, or only a declared compatibility window. An automated check without a policy can produce noise or false confidence.

Governance that fits the delivery system

API governance does not require a central committee to approve every field. It does require ownership and shared defaults. Assign an owner for each public or cross-team contract, publish its specification in version control, and make compatibility checks part of the normal pull-request path. Reserve human review for changes with broad impact: semantic changes, authorization changes, new versions, or removals that affect multiple consumers.

A practical release sequence might look like this:

• Update the contract and run linting plus backward-compatibility checks.
• Run consumer contract verification against the provider change and publish the verified contract revision.
• Deploy compatible consumers before changing producer behavior when a rollout requires both representations.
• Monitor error rates, unknown-field handling, version usage, and idempotency conflicts after release.
• Remove deprecated behavior only after the stated window and verified migration, not merely because the replacement has shipped.

Operational signals can reveal contract problems that tests miss. Track error rates by API version and client identity, but avoid high-cardinality metrics labeled with raw user identifiers. Log request correlation identifiers and idempotency outcomes carefully; keys may be sensitive operational data and should not become a source of unnecessary exposure. Alert on meaningful changes, such as a sharp rise in duplicate-key payload mismatches or calls to a version approaching retirement.

Governance should also define how exceptional changes are handled. Security fixes sometimes require a shorter deprecation window; a faulty field may need immediate rejection. Record the reason, affected consumers, mitigation, and rollback plan. Consistent exceptions are safer than pretending every change can follow the same calendar.

A review checklist for a consequential API change

Before approving a change to a heavily used contract, ask:

• Does it alter the meaning, units, default, accepted range, or failure behavior of an existing field?
• Can older consumers ignore new response data, and can the provider continue accepting older request shapes?
• For protobuf, have deleted field numbers and names been reserved?
• If the operation can be retried, are idempotency-key scope, retention, concurrency, payload mismatch, and error behavior defined?
• Do consumer contracts represent actual dependencies without asserting irrelevant response details?
• Are authentication, routing, deployment, and network behaviors covered by tests or operational checks outside the contract suite?
• Is there an owner, migration path, deprecation window, and measurable criterion for removal?

These questions turn “is this API backward compatible?” from a subjective judgment into a reviewable engineering decision. The answer may still involve trade-offs: a new version can impose duplication, a compatibility shim can prolong complexity, and a strict schema rule can block a legitimate feature. What matters is making the cost visible before consumers discover it through a failed request.