Ten models, two honest frames
Take a crowd of ten near-optimal models on CEH-001 under one frozen loss. Five sit in the spread frame and mark near 1.50. Five sit in the curve frame and mark near 2.10. This is not model failure. It is underspecification behaving exactly as the earlier chapters say it will: many models fit almost equally well, and they disagree because the frames disagree, not because the fits are poor.
Let an ensemble do what ensembles do. The crowd mean is (5·1.50 + 5·2.10)/10 = 1.80. The sample standard deviation across members is 0.316, so the standard error of the mean is 0.316/√10 = 0.100, and a 95% interval on the ensemble estimate is 1.80 ± 1.96·0.100 = [1.60, 2.00].
Read what just happened. The ensemble reports the forbidden midpoint, and it reports it with a tight interval that contains neither honest cloud — not 1.50, not 2.10. This is the most dangerous form of the midpoint sin in the whole book, because unlike a committee splitting the difference in a meeting, this one arrives with a standard error and the full aesthetics of statistical rigour.
The strongest objection, answered honestly
A careful reader should object here, and the objection deserves arithmetic, not a slogan. Suppose the desk later settles on the spread frame, so the realised mark is 1.50. Each spread member scores squared error 0; each curve member scores (2.10-1.50)² = 0.36. The average member's expected squared error is ½(0) + ½(0.36) = 0.18. The ensemble's squared error is (1.80-1.50)² = 0.09. Averaging halved the error. That is not a fixture accident; it is the ambiguity decomposition, which says the average member's error always equals the ensemble's error plus the variance across members: 0.18 = 0.09 + 0.09.
The ensemble genuinely is more accurate under squared-error loss against an unknown frame choice. So the real objection is not that the ensemble is wrong. It is that it is optimal for a loss nobody carded — a mixture loss with an implicit fifty-fifty prior over which frame gets chosen, approved by no owner, bound to no training hash. The desk's actual decision object is different: produce a mark under a named ruler, or escalate so a human names one. Under that object, 1.80 scores worse than useless, because it carries zero density under either frame's own honest law. If the desk wants to hedge the frame choice, the legitimate move is to hold both marks and size to survive either — that is what escalation with dual marks is. Averaging is not a hedge. It is a silent position in a third world, guaranteed wrong under whichever ruler eventually wins, differing only in how wrong.
The gate is DUAL_MARK_MIDPOINT, and a tight standard error does not soften it. Members disagreeing loudly is a signal to escalate. Members agreeing quietly on the wrong number is worse, and harder to catch, because it looks like confidence.
A second way to reach 1.80: buying a cue
A different ensemble, forty boosted trees this time, reports exam mean absolute error of 0.061 against a champion's 0.084, a 27% improvement, and its marks on the friendly slice cluster tightly between 1.79 and 1.82. It looks like the model actually resolved the frame disagreement. Ablate one feature, quote intensity, and the marks reopen to 1.49 and 2.11, a gap of 0.62. The tight cluster around 1.80 was never the model settling the frames; it was one cue collapsing them. Worst-group error confirms it: 0.052 on the friendly slice, 0.191 on the co-break slice, a 3.7× ratio against a 2.0× limit.
Same number, 1.80, reached by a completely different mechanism — not averaging across frames this time, but a feature that suppresses the frame distinction inside every member. The gate has a different name, SHORTCUT_CUE, and a different remedy, because the value of a number never diagnoses its cause. A model that "almost" resolves this once the cue is understood is disqualified, not yellow.
Why estimators live in this book at all
Statistical decision theory, shrinkage, bandits, sequential estimators — none of it is here as a curriculum this book respects for its own sake. Each idea earns its place by the same test: can it be written on a card and made to refuse a spend? A regularisation technique that cannot become a field on a loss card has no home here, however well it scores in a benchmark. Shrinkage intensity is a loss-card field, not a tuning knob left to memory — shifting it from 0.35 at training to 0.10 at exam has been measured to move a reported portfolio volatility by roughly 71 basis points in the flattering direction with no change to any model, which is exactly the kind of drift a card, not a memo, is supposed to catch.
Sequential rights: when an exploration bonus should not buy you size
Bandits give the Permit ladder a vocabulary, and vocabulary needs a collision to prove itself. Run UCB1 at t = 100 total pulls. The A2 shadow arm has averaged 0.42 across 60 pulls: its exploration bonus is 0.392, giving it an index of 0.812. The A4 sized-spend arm has averaged 0.55 across just 3 pulls: its bonus is 1.752, more than three times its own observed mean, giving it an index of 2.302.
UCB says pull A4, and says so emphatically. But that bonus of 1.752 is thirty-five times the desk's own pre-registered minimum detectable effect of 0.05. It is not evidence. It is ignorance, priced as opportunity, wearing the units of reward. The harness does not argue with the index — it checks whether A4's rights were ever granted. The exam behind it returned inconclusive (a twelve-day fixture with a confidence interval of [-0.04, +0.08] against an MDE of 0.05), so A4 is not in the action set at all. There is nothing for UCB to select.
The deeper reason is not a patch to one algorithm. Regret minimisation optimises a sum over pulls, and it is right to do that when losses are bounded and comparable — a bad research replay costs compute, a bad shadow observation costs a log entry. At A4 the condition fails: a good sized pull is worth +0.55 in the table's units, but a sized pull on a leaked or coupled model costs -50, a family retirement, a restated book, an audit. Accept three exploratory sized pulls per hundred to learn faster and the expected value is 97·0.55 + 3·(-50) = 53.4 - 150 = -96.6, reported correctly as regret-bound-honouring the whole time. Ninety-nine good outcomes do not compensate one that removes the desk's licence to hold the position at all.
Off-policy evaluation: five hundred decisions, seventy-one of them real
A desk holds 500 logged decisions and wants to evaluate a new policy off-policy, reweighting each outcome by how much more, or less, likely the new policy was to have taken that action. Suppose 480 of those weights come out at 0.5 and 20 come out at 13 — they sum to 500, exactly as they should, so a careless read calls this a 500-observation study.
The effective sample size says otherwise: ESS = 500² / (480·0.25 + 20·169) = 250,000 / 3,500 = 71.4. Detectable effect size scales like 1/√n, so the minimum effect this study can actually resolve is √(500/71.4) ≈ 2.65 times larger than advertised. A pre-registered MDE of 0.05 is, on this pack, effectively an MDE of about 0.13 — and every result that looked significant somewhere between those two numbers was never in range. The fix is a card field, not a caveat: log the propensity of every action, report effective sample size next to nominal sample size, and compare the MDE against the effective figure, not the flattering one.
Regulatory language names the same gate: three yellow columns, one book
Basel's own traffic-light backtest turns out to be the harness's family-coupling gate wearing different stationery. Over 250 days a 99% VaR model expects 2.5 exceptions. CEH-001 posts 8 (yellow, multiplier 3.75), Cousin-A posts 7 (yellow, 3.65), Cousin-B posts 9 (yellow, 3.85). Reviewed one name at a time, this is three uncomfortable quarters and nothing more — no name is red, and every desk head can defend their own column.
Now look at the dates instead of the counts. All three names took their exceptions on the same six days, driven by one shared scheme break inherited by all three models. The correlation across the family is the finding; the individual counts were never the finding, and single-name monitoring is structurally incapable of seeing it. Action: BOOK_COUPLING. Retire the family, freeze the shared cue's retrains, open escalate-only research until a book-scope challenger clears a joint exam. A regulator would call this correlated model risk requiring independent review. The harness calls it family retirement. They are one action wearing two names.
One more habit is worth naming here because it is where supervisory guidance is usually misread: desks tier a model inventory by sophistication — linear models low risk, anything with a neural network high risk. That is the wrong axis. Tier by rights class instead: a model that can reach sized spend is top tier regardless of how simple its math is; a model capped at shadow is low tier regardless of its parameter count. Validation effort should land where spend can actually happen, not where the model looks impressive.
Book: Chapter 13 in Rulers Before Models.