Model-Agnostic by Design: Why We Don't Bet on One Provider

Introduction

In the last eighteen months, every major model provider has shipped at least one release that changed default behaviour in a way that broke production systems. Formatting changed. Refusal thresholds shifted. Output verbosity increased. Structured outputs started including fields that weren't in the schema. In most cases, the providers documented the changes. In most cases, teams found out when users started filing tickets.

The teams that handled these transitions smoothly had one thing in common: their application code did not know which model it was talking to. The model was a configuration value, not an assumption baked into the logic.

The coupling problem

Most AI application code is more tightly coupled to a specific model than it looks. It's not always explicit. You rarely see if model === "gpt-4" in a codebase. The coupling is subtler — in prompt templates written to exploit a specific model's quirks, in output parsers tuned to a particular response format, in temperature settings chosen because they worked well in a specific evaluation run that was never repeated.

When the model changes — even a minor version bump — these implicit assumptions break. And because they were never written down, they are hard to find and harder to test.

How we structured our adapter layer

We built a thin adapter interface that every model call goes through. It has four responsibilities: normalize the request format, inject model-specific prompt adjustments, normalize the response format, and emit a structured log entry with the model identifier. Application code above the adapter never references a model name or provider SDK directly.

Swapping models is a one-line environment variable change. Prompt adjustments specific to a new model version live in the adapter, not scattered across the codebase. When a provider ships a breaking change, the blast radius is a single file.

Maintaining behavioural parity across models

The adapter layer handles structural coupling. Behavioural parity is a separate problem — and it requires tests. When we evaluate a new model or model version for a use case, we run the full behavioural regression suite against it before any production traffic touches it. The suite defines the expected behaviours. The model either passes or it doesn't.

This has two benefits beyond the obvious. First, it forces you to articulate what "correct behaviour" means in a form that is testable — not just "it should answer helpfully" but specific input–output pairs for every category you care about. Second, it gives you an objective basis for provider decisions that isn't "we've always used X."

• Define expected behaviour first — write the tests before you evaluate any model

• Run evals on every model version bump — treat it like a dependency upgrade

• Keep adapters thin — business logic above, provider quirks below

• Use feature flags for gradual rollout — shadow-traffic a new model against the old one before full cutover

• Track provider-specific costs and latency separately — the cheapest model for your use case may not be the one you started with

The compounding benefit

Teams that build this way find that the model decision becomes genuinely competitive. When a new model ships, they can evaluate it against their actual workload in a few hours and ship it the same week if it passes. Teams that are tightly coupled wait for a sprint, schedule a rewrite, and ship it six weeks later — by which point something else has changed.

Model-agnostic design is not about hedging. It is about moving at the speed the AI ecosystem actually moves, which is fast and shows no signs of slowing down.