Guardrails Are Not a Feature. They're Infrastructure.
Bolting on safety after the fact is how you get incidents. Here's how to bake validation in from day one.
Jessica Beckett
-
Tech Lead
9 mins read
Why "bolt-on safety" fails
When safety is a feature, it lives at the edges of your system. A filter on the way out. A regex check on the way in. A moderation endpoint you call after the model has already responded. These work until they don't — and when they fail, they fail silently, at scale, in ways that are hard to trace back to a single point of failure.
Infrastructure is different. Infrastructure is load-bearing. You do not retrofit plumbing into a finished building. You route the pipes first, then build around them.
"A filter on the output is a fire extinguisher. Validation built into the pipeline is a sprinkler system. One of them requires someone to be in the room when things go wrong."
The deeper problem is that bolt-on safety creates a false sense of coverage. Your moderation call returns safe: true and the system logs it as a pass. But the model already completed the response. The user already read it. The damage, if any, is done. Filtering after the fact is not safety — it is incident documentation.
What infrastructure-first actually looks like
Building guardrails into infrastructure means validation happens at every layer of the request lifecycle, not just at the perimeter. Here is the architecture we use and recommend.
Layer 1 — Input schema enforcement
Before any token touches a model, validate the shape of what you're sending. This is not about content moderation yet — it is about structure. Malformed or unexpectedly large inputs are one of the most common sources of unpredictable model behaviour.
This sounds obvious. Most teams skip it. They trust that the prompt template will always produce well-formed input. It won't. Users will find edge cases you never imagined, and a schema layer is the cheapest line of defense you will ever write.
Layer 2 — System prompt pinning
Your system prompt is a contract between your application and the model. Treat it like one. Do not allow user-supplied content to bleed into the system turn. Do not interpolate untrusted strings directly. Pin the structure and validate any dynamic segments before they go in.
Layer 3 — Structured output enforcement
If your application depends on a specific output format — JSON for a downstream API, a specific schema for a UI component, a bounded set of values for a decision — enforce it structurally, not by parsing free text. Use JSON mode or tool-use to constrain the output shape at the API level, then validate the response against the same schema your consumer expects.
This eliminates an entire class of runtime errors: the model confidently returns a response that looks right, parses fine, but has a missing key your code assumes is always present. Schema validation at the response boundary catches these before they become user-facing bugs.
Layer 4 — Semantic guardrails in the prompt layer
This is where most teams start and stop, but it is actually the fourth layer, not the first. Semantic rules — what the model should and should not say, how it should handle edge cases, what it should do when a request is out of scope — belong in the system prompt as explicit, tested instructions. Not vibes. Not hopes. Instructions with examples of both correct and incorrect behaviour.
Write them like unit tests: for each class of behaviour you care about, write an example prompt and the response you expect. Then test against those examples every time the system prompt changes.
Layer 5 — Runtime observability
Guardrails you cannot observe are guardrails you cannot improve. Every completion your application makes should emit a structured log entry that includes the input hash, the output category, any validation errors, latency, and cost. Not so you can watch a dashboard — so that when something goes wrong, you have a queryable record of exactly what the model was sent and what it returned.
The organizational dimension
None of this is technically difficult. The hard part is organizational. Guardrails-as-infrastructure requires that safety work happens before, not after, a feature ships. That means product and engineering need a shared definition of what "done" means for an AI feature — one that includes validation layers and observability hooks as requirements, not nice-to-haves.
It also means treating changes to system prompts and model versions with the same rigor as changes to application code. A new model version is a dependency update. A revised system prompt is a configuration change. Both can change behaviour in ways that your existing tests will not catch — unless you have written tests for the behaviours that matter.
"Ship with the logs you wish you'd had the last time something broke. Because something will break."
A checklist for day one
If you are starting a new AI feature today, here is what should be in place before you write the first completion call:
Input schema defined and enforced — shape, size limits, and type constraints
System prompt under version control — changes are reviewed, tested, and logged
Output schema defined — if the consumer expects a structure, validate it structurally
Behavioural test suite — at least 10 examples per guardrail category, run on every prompt change
Structured completion logging — every call emits a structured log with the fields listed above
A fallback path — when the model fails validation, returns an unexpected output, or times out, your application degrades gracefully rather than surfacing a raw error or a hallucinated response to the user
A defined escalation path — when the logs surface anomalies, who owns the investigation ?
None of this takes more than a day to set up on a new project. Retrofitting it into a production system that was never designed for it takes weeks, happens under pressure, and never quite fits the way it would have if it had been built in from the start.
Guardrails are not a feature. They are the ground the feature stands on. Build them first.