The problem we left open
In the last post I used bolted guardrails to name a failure mode that demo culture keeps reproducing.
Train freely. Impress a room. Wrap filters at the end. The gateway looks serious. The decision chain is still missing. On CEH-001, you can redact PII and still ship midpoint peace, promote on unpowered calm wins, and train on features that never should have entered the room. Serving controls are not a harness. They are one layer of one harness.
That creates three real headaches in production:
- Shift-right safety: the only hard checks arrive after weights and habits already exist.
- Category error: filters mistaken for decision geometry, eval power, and rights design.
- Demo-path lock-in: advisory controls that must not break the happy path.
So the question for this post is simple. If that is the failure mode, what does a real AI solution look like?
The solution, as one stack
The core idea: make harness the product word. A harness is the automated path that makes the earlier objects unavoidable: quality bars, loss and information rules, powered evals, serving rights that cannot bypass Clearance, and living documentation tied to monitors. Five moves, one stack. Not five optional add-ons you schedule after launch.
1. Data and feature quality gates before train
Define expectations that can fail closed: schema, as-of integrity, join parity, leakage canaries, forbidden cue lists, freshness, and population tags for calm versus hard weeks. A batch that fails does not enter the training registry. This is shift-left in the literal sense. You stop poison before optimization memorizes it. For CEH-001 that includes refusing labels born as panic midpoints or unverified agent marks unless re-labeled under contract.
Publish the expectations next to the feature code, not in a wiki nobody runs. If a check cannot block a registry write, it is journalism about data quality. Journalism does not stop shortcuts from entering weights.
2. Bind training to a loss card and an information contract
No anonymous objective. No anonymous information room. The training job must point at a versioned loss card and a versioned information contract. The harness refuses to start if either is missing, stale relative to the feature build, or incompatible with the typed job the model will serve. Prompts can still help. They do not replace those cards. Midpoint peace cannot be an undeclared target because the card never named it as allowed.
This is where bolted culture usually tries to cheat: train first, attach cards later as paperwork. The harness inverts that. No card ids, no trainer process. Paperwork that cannot start a job is not paperwork. It is control.
3. Keep the eval harness in the promotion path
Use the powered-claim stack: frozen pack, paired deltas with uncertainty, minimum detectable effect, selection correction, CI gate. Promotion consumes those artifacts or stays escalate-only. A serving filter must not be allowed to waive an eval fail. That waiver pattern is how bolted systems recreate themselves inside a prettier diagram.
Hard slices stay first-class here. A calm-only green is not a CEH-001 promotion input. If the never-traded family is the product, the exam that can flip a calm champion has to sit in the same gate that opens tiny size.
4. Serving gateway controls that cannot bypass Clearance
Allowlists, rate limits, PII handling, and tool rights still matter. Put them at the edge. Bind them to Clearance tokens so a clean gateway call without an allow token does not spend. The gateway enforces communication and tool hygiene. Clearance enforces decision rights. Neither substitutes for the other. An agent that passes PII checks and lacks a mark token still fails closed.
I want that split boringly explicit in the architecture diagram. Edge hygiene on one side. Decision rights on the other. Teams that draw only the edge will keep believing a redacted paragraph is a validated mark.
5. Living model card plus monitor hooks
The card is not a PDF cemetery. It names frames, loss, information rules, eval pack hash, known limits, and the monitors that can revoke autonomy. Hooks fire into the same degrade path you trust for live shift. When the card and the running system disagree, that is an incident, not a documentation backlog. Harness means the happy path updates the card or refuses to ship.
Put together: quality gates, training bound to cards, powered eval promotion, gateway under Clearance, living card with revoke hooks. That is a full harness. Filters are inside it. They are not the definition of it.
The example: CEH-001 on the happy path
A team wants tiny auto size on the never-traded family. Dual frames still sit near 1.50 and 2.10. Old demo culture would wrap filters and open size. Watch the harness refuse that shortcut without a speech.
Monday: a feature build fails an as-of canary and a forbidden-cue check. Training never starts. Tuesday: a clean build points at a loss card that scores frame-aware error and forbids silent midpoints, plus an information contract that pins clocks. Training runs. Wednesday: challenger B looks plus two on calm. Eval harness refuses promotion: interval overlaps zero, hard slice flips, search size was nine. Thursday: a narrower candidate passes powered gates. Gateway allowlists and PII controls are already on, but spend still needs the Clearance mark token. Friday: model card publishes pack hash, dual-frame policy, and monitor hooks. A coverage slip next month can revoke without waiting to invent process.
End state: the demo still exists, but it cannot skip the building code. 1.80 remains deny-by-default. Filters never got to pretend they were the whole safety story. The harness made the boring objects the shortest path to production, which is the only way they survive contact with a calendar.
That is what I mean by harness as product word. Not a metaphor. An automated path where skipping a loss card, an eval interval, or a Clearance token is harder than doing the work. If skipping is easier, you still have a demo with gift wrap.
The flow in one breath
Problem: bolted filters after a demo leave the decision chain unfinished. Solution: one harness that blocks bad data before train, binds learning to loss and information cards, powers eval gates, keeps serving controls under Clearance, and maintains a living card with revoke hooks. Example: CEH-001 tiny size only after those objects already exist on the happy path.
Curious how others make harness the default path without turning every change into ceremony, and where they still see teams confuse a gateway deny with a finished system.