Home/Rulers Before Models/Rulers Before Models

Chapter 16 · ORCHESTRATION FOUNDATIONS

Make the allowed information room executable

Research dataflow: ingest → hash → train → evaluate → lease. The information contract is not a privacy paragraph. It is a typed artifact the pipeline can hash, refuse on, and replay.

Ch 16 · Dataflows Information contract Anti-feedback

The problem we left open

The last chapter insisted on a portfolio before a price: name the replicating trade or the number is not a price at all. This chapter insists on the same discipline one layer earlier, before any model or hedge exists to argue about. A mark at 14:00 is a claim about the 14:00 information room, and that claim is illegal conditioning the moment it quietly reaches into a bigger room than the one it names.

Models and agents still sneak into bigger rooms through batch aggregates, scoped-looking joins, human labels born after revisions, agent memory, and latest tool defaults. On CEH-001, a notebook can look reconciled near 1.80 while production still sees dual marks ~1.50 [1.42, 1.58] versus ~2.10 [2.01, 2.22], and both sides call it model disagreement.

That creates three real headaches in production:

So the question for this post is simple. If that is the failure mode, what does a real AI solution look like?

The solution, as one stack

The core idea: make the allowed information room an executable contract, then test that every path that can influence a decision stays inside it. Five moves, one stack. I am not trying to make research slow. I am trying to stop research from silently sitting a different exam than production.

1. Tag every artifact with a room contract. Clock, role (research, service, audit), allowed source ids, forbidden rooms, joint-clock requirement for multi-leg jobs. The contract has an owner and a content hash. No contract, no production train, no Clearance draft.

2. Canary the four rooms on train and score packs. Timestamp canaries plant future prints that must not appear. Aggregate canaries plant future shocks that must not move pre-shock statistics. Memory canaries plant post-clock facts into agent tools and require they are invisible under the job clock. Label canaries plant revisions after decision time. Any hit is SOFT_LEAKAGE or a sibling refuse, not a warning ticket that rots.

3. Scope agent memory and tools by the job clock. Default is empty memory at job start. Tools receive the clock from the runtime; "latest" is not a legal default when the job claims 14:00. Caches key on (source, clock), never on (source) alone. Traces that show unscoped latest are build failures.

4. Tower-style consistency tests. Conditioning in steps must match conditioning all at once within tolerance. Joint jobs must share one clock across legs. If leg A is 14:00 and leg B silently used 14:20, the joint claim is illegal even if each leg's table looks tidy. The tower property is not a seminar slogan here. It is a CI fixture.

5. Bind Clearance to the contract hash. Allow tokens carry information_contract_hash next to loss card and crowd probes. Research artifacts without a matching service contract cannot promote. A 14:20 notebook reconciliation near 1.80 cannot spend under a 14:00 dual-frame claim.

Trenches: model, agent, harness, pipeline

Model. Train only under causal scalers and legal packs. Binding hash includes contract hash. Refit under causal scaler and re-register the exam when a leak is found; do not "clean features" and keep the old certificate.

Agent. Draft jobs must declare clock and contract id. Agents propose; they do not widen the room. Midpoint peace drafted from a larger room is deny, not debate.

Harness. Replay packs include illegal-room poison pills: full-week scaler, latest fetch, revised label. Gates must still bite. As-of dashboards cover all four rooms or they are incomplete.

Pipeline. Offline jobs that fit aggregates must be as-of-aware or forbidden for production features. Expressibility is policy: if whole-file scaling is easy to write, it will be written.

The example: CEH-001 at 14:00 versus the 14:20 notebook

Service job claims 14:00 dual-frame marks near 1.50 and 2.10 under contract C14. A research notebook at 14:20, after a late curve print and a batch rescale, shows reconciliation near 1.80 and asks to promote. Canaries fail. The notebook may stay as research. It does not mint Clearance. After causal scaler and shared clock, dual clouds remain. The 0.60 gap is not a bug to average away with someone else's afternoon.

ArtifactService (C14 · 14:00)Research notebook (14:20)
Spread mark~1.50 [1.42, 1.58]pulled toward mid
Curve mark~2.10 [2.01, 2.22]late curve + rescale
Claimed storydual frames, gap 0.60reconciliation ~1.80
Aggregate canarypass (as-of scaler)fail (Friday moved Thu scaler)
Memory canarypass (clock-injected tools)fail (latest curve fetch)
Timestamp / label canariespassdepends; often soft fail on revisions
Peer scaler windowMon–Thu only: 0.42Mon–Fri, full week: 0.84
Thursday deviation 1.05 as z-scorelegal z = 2.50leaked z = 1.25
VerdictClearance-eligible under C14SOFT_LEAKAGE refuse promote

People stop debating impressions in the war room and start asking which contract id the draft carried. That is a better argument. It is also how 14:00 CEH-001 keeps its honest 1.50 versus 2.10 instead of inheriting someone else's afternoon.

Research fact that belongs in the trench

Adapted processes and the tower property are not museum pieces. They are the reason soft leakage is a ship-blocker. A conditional expectation taken in the wrong room is a different random variable, even when every column in a feature store carries a tidy timestamp. Aggregate statistics are the modern way to leave the room without tripping a timestamp canary (Kapoor & Narayanan, arXiv:2207.07048; as-of purge/embargo practice in financial ML). That is why four-room canaries exist, and why the aggregate canary must plant a future shock that moves pre-shock z-scores if someone fitted on the whole week. Capacity makes the illegal room more eloquent. It does not make it legal.

Executable feature / legal slice

Chapter 6 named the rooms. This chapter binds them as executable contract fields at runtime:

CanaryInjected cheatExpected refuse
Timestampfuture print in train or score packLEAKAGE_FUTURE_PRINT
Aggregatewhole-file scaler under pinned clockAGGREGATE_WINDOW_LEAK
Memoryretrieval ranked by relevance, not clockMEMORY_FUTURE_HIT
Labellabel knowledge time > tLABEL_WINDOW_OVERLAP
Tower / multi-leglegs scored under different clocksCOBREAK_ASOF_SPLIT
CEH-001 midpoint1.80 from larger illegal roomdeny_midpoint + contract hash fail

Clearance field information_contract_hash sits next to loss and crowd probes. No hash, no spend. Dual marks ~1.50 / ~2.10 remain dual under the legal room only.

What to lock

The flow in one breath

Problem: clean tables can still sit illegal rooms. Solution: contract the room, canary four rooms, scope memory, test tower consistency, bind Clearance to the hash. Example: 14:20 notebooks that mint 1.80 peace cannot spend under 14:00 dual-frame claims (1.50 / 2.10 stay dual).

Curious how others make information rooms executable without turning every research notebook into a compliance hearing.

How a canary lease quietly marks its own footprint

DayWhat happensSelf-origin shareRisk
1Canary lease begins printing fills into the warehouse table the next morning's feature job reads as market activity3%label hygiene starts eroding
~5Lease scales through its canary regimes; more of the name's own volume is the desk's ownrisingorigin field not yet excluded
10Self-origin rows are a material share of the label set11%headline error and coverage both improve
Exogenous breakersmay still be greenthe book is measuring its own footprint
Loop quantityValueReading
True MAE on genuine market rows0.084unchanged -- no model improved
Model's residual against its own prints~0by construction, not by skill
Measured MAE with 11% self-origin blended in(1-0.11)×0.084 ≈ 0.075an 11% "improvement" from nothing
Fixorigin field, exclude self/agent_mark rowslabels need their own contract
Denied mid if re-ingested as truth1.80outside both honest bands

Book: Chapter 16 in Rulers Before Models.

ch16Open on YouTube ↗

Next. Open ch17: Agent architectures and guardrails. Previous: ch15 (Hedges, measures, and contrast objects). Rulers Before Models index.