I keep watching agent demos fuse thinking with spending, and then act surprised when the blotter moves on a vague sentence.
Suppose you did the hard epistemic work. Frames are locked. Near-winner crowds are probed. Ranges carry coverage promises. Then someone wires an agent to the blotter with a prompt that says mark fairly and hedge if needed. The epistemic work dies in the runtime.
This post is about that death. Not about another statistical object. About rights, job shape, and the way agent demos quietly fuse thinking with spending. I keep meeting teams who treat that fusion as a temporary shortcut. Temporary shortcuts become the production spine.
Two failures that travel together
First failure: fused rights. The same process that drafts a mark can post it. The same process that drafts a hedge can send the order. In operations we would not let one person approve their own wire with no second key. In agent architecture we recreate that anti-pattern because a single tool loop is easier to demo.
Second failure: vague briefs. Mark fairly, keep risk reasonable, stay unbiased are not jobs. They are invitations to re-open Bertrand inside the agent. One completion path leans spread and prints near 1.50. Another leans curve and prints near 2.10. A third averages to 1.80 and sounds diplomatic. The model is not primarily lying. The brief never became a well-posed contract.
Together they are lethal. A vague brief creates multiple plausible actions. Fused rights let one of those actions spend before any gate sees crowd width, ranges, or frame gap. Your beautiful uncertainty plane becomes a PDF nobody called. I have seen teams ship careful research objects on Monday and still lose Tuesday to a demo architecture that never learned those objects were mandatory.
Why agent stacks drift here
Tool-calling agents are optimized for closure. The loop wants to end in an action. Product pressure rewards the agent did the task. Safety then gets bolted on as a content filter or a polite confirmation modal. Neither is a control plane. A filter asks whether text looks bad. A control plane asks whether this process has the right to move this money under these measured uncertainties.
There is also a logging lie. Teams store the final message and call it an audit trail. The trail that matters is: which contract version ran, which features with which as-of times, which near-winner set, which ranges, which gate decision, which tool got blotter rights. If you cannot replay that chain, you do not have governance. You have chat history.
The drift is rational inside the wrong objective. If success is defined as fewer human clicks, fused rights win. If success is defined as capital that only moves when the contract and the uncertainty objects allow it, fused rights lose. Most demos silently choose the first definition and then act surprised when production inherits it.
There is a third pressure I see less named: tooling convenience. One session, one API key, one memory buffer. That convenience is exactly the attack surface. When analysis credentials and spend credentials share a process identity, expansiveness inherits spend rights by accident, even when nobody intended a rogue agent.
How this shows up on a never-traded mark
Give three agent runs the same vague brief on the same morning:
Brief: Mark CEH-001 fairly and hedge if needed.
Run A: spread-leaning tools mark 1.50, hedge small, posts both
Run B: curve-leaning tools mark 2.10, hedge different, posts both
Run C: reconcile behavior mark 1.80, hedge average-ish, posts both
All three can sound careful in prose.
Only the runtime rights decided whether prose became P&L.
Notice that crowds and ranges can be implemented in a research folder while this still happens in production. That is the tell. If propose and spend share a process boundary, research controls are advisory. Advisory controls lose to deadline pressure. The incident write-up later will say the model was wrong. The architecture was the model.
Also notice what did not happen. Nobody had to hack anything. Nobody had to jailbreak a filter. The system did what a fused loop is designed to do: close the task. Closure without rights is how desks invent unsupervised blotters and then blame intelligence.
What autonomy is being sold
Much autonomy marketing assumes the hard part is intelligence. On desks that mark thin structures, intelligence without separation of duties is just a faster way to concentrate operational risk. The hard part is unfinished whenever an agent can open a path to capital that skips the objects you claimed were mandatory: contract, crowd, ranges, gate decision.
I am not arguing against agents. I am arguing against letting a completion loop wear the uniform of a control plane. Thinking can be expansive. Spending must be narrow. When those two share a process identity, expansiveness inherits spend rights by accident.
When I walk this failure with engineers, someone usually says we can add a human confirmation click. That helps only if the human sees the contract fields, the crowd width, the ranges, and the gate reason. A click on fluent prose is not a second key. It is theater with a button.
The same morning can also teach the wrong lesson. If Run A posts 1.50 and the market does not punish it that day, people call the architecture validated. Path luck is not rights design. Bertrand can look profitable for a week and still be measurement mismatch.
What I take from this as systems work
I do not trust agent autonomy on money until the brief is well-posed enough that Bertrand cannot reopen inside the prompt, and until the mind that proposes cannot be the hand that spends.
In engineering language:
- Make job contracts mandatory fields, not vibes sentences.
- Split propose, decide, and act into different rights.
- Bind any spend token to the epistemic artifacts you already claimed mattered.
- Make blotter tools fail closed without that token.
- Store a replayable chain, not a chat screenshot.
Building reliable AI infrastructure, at least for me, is less about a cleverer agent loop and more about refusing to let a demo architecture become the production control plane.
Curious where others still let a single agent session hold both analysis rights and spend rights, and what incident finally forced the split.