Home/AI Orchestration · Model Risk/Part I

Episode 30 · PROBLEM · Ops / harness

Separate models are not independent risk

Why a book of green single-name gates can still break as one bet

Name-by-name greens hide one joint bet when cousins share features and calm years.

Name-by-name greens hide one joint bet when cousins share features and calm years.
flowchart TB
 subgraph MARGIN["Name-by-name greens"]
 A["CEH-001 green"]:::input
 B["Cousin A green"]:::input
 C["Cousin B green"]:::input
 D["Dashboard row of greens"]:::artifact
 end
 subgraph COUPLE["Hidden joint bet"]
 E["Shared features / loss / calm years"]:::risk
 F["Same single-name eval pack"]:::risk
 G{"Joint co-break measured?"}:::decision
 end
 subgraph FAIL["Correlated failure"]
 H["Ugly week hits whole family"]:::risk
 I["Diversified story = one bet"]:::risk
 J["1.50 vs 2.10 gap moves as a bundle"]:::risk
 end
 A --> D
 B --> D
 C --> D
 D --> E --> F --> G
 G -->|no| H --> I --> J
 classDef input fill:#CCFBF1,stroke:#0F766E,color:#134E4A,stroke-width:2px
 classDef decision fill:#FEF3C7,stroke:#B45309,color:#78350F,stroke-width:2px
 classDef risk fill:#FEE2E2,stroke:#B91C1C,color:#7F1D1D,stroke-width:2px
 classDef gate fill:#DCFCE7,stroke:#15803D,color:#14532D,stroke-width:2px
 classDef process fill:#E0E7FF,stroke:#4338CA,color:#312E81,stroke-width:2px
 classDef artifact fill:#F5F5F4,stroke:#57534E,color:#1C1917,stroke-width:2px

Most teams I watch still gate models name by name. CEH-001 looks fine. Cousin A looks fine. Cousin B looks fine. The dashboard is a row of greens. Then one ugly week hits the whole structure family at once, and the "diversified" book behaves like one bet with many tickers.

The quiet idea underneath that week is book coupling: models that were trained, scored, and promoted as separate objects can still fail together. Separate training runs are not the same thing as independent risk. Independence is a claim about joint behavior. A folder of single-name certificates never measures that claim.

What "separate models" quietly pretends

In probability language, two random variables can each look well behaved on their own and still move as a bundle. Correlation, shared factors, common regimes, shared loss shapes: any of those can make the joint story worse than the marginal story. Finance has lived with that forever in portfolios. Financial AI often forgets it the moment models get their own repos and their own green lights.

Independently trained does not mean independently wrong. If every model in a family optimizes a similar objective, drinks from a similar feature kitchen, sees the same calm years in training, and faces the same single-name eval pack, they are not five opinions. They are five copies of one opinion with different random seeds and slightly different names. That is evaluative compression at book scale: the exam itself squeezes many candidates toward the same shortcuts and the same blind spots.

There is a second confusion that makes this worse. Teams hear "ensemble" and feel safer. An ensemble of coupled learners is not insurance. If the members share the fragile assumption, averaging them can sharpen the wrong answer. Diversity of filenames is not diversity of failure modes.

I like to say it bluntly. A gate that only asks "is CEH-001 acceptable?" can miss the question the blotter will ask: "what happens to the family when the shared driver flips?"

How this shows up in production

1. Single-instrument courage. Each structure clears its own stress pack. Joint stress across the book never runs, or runs as a polite slide that nobody can fail a release on. The desk thinks it bought diversification because the names differ. It bought correlated mark error with a spreadsheet of greens.

2. Shared-driver blindness. One liquidity cue, one funding proxy, one calm-regime quote intensity feature shows up across cousins. Each model looks sensible alone. On the co-break week the cue dies everywhere at once. Errors arrive as a chorus, not a solo.

3. Shared-objective cloning. Every family member was tuned on a loss that rewards the same peace midpoint or the same proxy PnL. Near-winner crowds inside one name already warned about that. Across names it becomes synchronized theater: many models, one wrong peace.

4. Eval-criteria compression. Promotion uses the same calm pack, the same hard-pack nickname, the same "+2 points wins" culture for every cousin. Selection pressure invents a family style. Later, when the world moves, the family fails in that style together.

5. Retirement that never fires at book scope. One name's monitors go red and someone debates a patch. Three cousins go red the same week and the org still treats them as three local incidents. Nobody owns the coupled failure as a book event.

A walkthrough on CEH-001 and its cousins

Take the never-traded family again. CEH-001 can still show the old dual-frame gap: one allowed setup near 1.50, another near 2.10, with the dishonest midpoint humming around 1.80 when people get tired. Now put three cousins beside it. Same structure grammar. Slightly different underliers or tenors. Each has its own champion model. Each champion cleared a single-name gate last quarter.

Under the hood they share more than the desk admits. They lean on the same liquidity intensity cue that correlated with fair marks in quiet years. They share a loss shape that punishes frame gap visibility and rewards a compromise number. Their eval packs stress one dial at a time and never ask what happens when funding, spreads, and quote intensity break as a bundle across the whole family.

Quiet months look like diversification. CEH-001 marks a managed band. Cousin A and Cousin B do the same. Limit usage looks spread out. Risk reports celebrate "multiple models."

Then the co-break week arrives. Quote intensity collapses across the family. The shared cue that propped every champion flips together. CEH-001's peaceful 1.80 culture tries to reappear as panic average while the honest dual frames reopen toward 1.50 and 2.10. The cousins do not hedge that pain away. They rhyme with it. Correlated mark errors stack. What looked like three independent leases becomes one book-sized miss with three line items.

The painful part is managerial, not mathematical. Nobody lied on the CEH-001 scorecard. The scorecard never asked the joint question. Probability dependence returned at book scale, and the governance language was still speaking in singular nouns.

If you only remember one diagnostic, remember this: when three cousins miss in the same direction on the same afternoon, do not open three unrelated model tickets first. Open the coupling question first. Ask which driver, which objective, which eval pressure made them rhyme. Local patches on synchronized failure are how the next co-break week gets scheduled.

What I take from this as systems work

I treat "separate models" as an organizational convenience until a joint stress pack and a coupling map prove otherwise. In engineering language that means a few dull habits:

Without those habits, autonomy on a "diversified" AI book is still a single-instrument story wearing many labels. Curious how others catch correlated model failure before a co-break week teaches the desk that separate greens were never independent risk.

Next. Open Ep31: A book harness: joint stress, budgets, and retirement. Previous: Ep29 (Make the harness the product, not the demo wrap). Part I index.