This chapter is Arc I's last mechanism: temperature, top-p, and min-p - the control surface between softmax logits and the tokens your verify pass (Appendix F.3) must audit. At inference, the language model outputs a probability distribution over the next token. The choice of how to sample from this distribution dramatically affects output quality. This chapter covers the three standard sampling parameters: temperature, top-p, and min-p.
Figure 1. Concept map for this episode: fight, desk translation, what to lock.
Temperature
Temperature is a scalar applied before the softmax that scales the logits. At a temperature of 1, the distribution is the model's native output. Below 1, the distribution becomes sharper - high-probability tokens become even more likely. Above 1, the distribution flattens - low-probability tokens become more likely. As temperature approaches zero, the distribution becomes deterministic (always picking the single most likely token); as it grows very large, the distribution approaches uniform randomness. For factual or structured outputs (JSON, code, numerical answers), low temperature (around 0.1-0.3) is appropriate. For creative or open-ended outputs, higher temperature (around 0.7-1.0) produces more diverse outputs.
Top-p (nucleus) sampling
Even with temperature applied, the probability distribution still includes a long tail of very-low-probability tokens. Top-p sampling (also called nucleus sampling) truncates the distribution to only include the tokens whose cumulative probability exceeds a threshold (e.g., 0.9). At a threshold of 0.9, the model considers only the smallest set of tokens whose combined probability is at least 90% - typically 5-50 tokens at any given step. The remaining tokens are zeroed out and the distribution is renormalised.
Min-p sampling
A more recent improvement is min-p sampling, introduced in 2024. Instead of fixing a cumulative probability threshold, min-p sets a floor relative to the most-probable token. A token is kept only if its probability is at least some fraction (typically 0.05 to 0.10) of the probability of the most-probable token. The intuition: in a sharply-peaked distribution (one token has 80% probability), the long tail is irrelevant and should be trimmed aggressively. In a flat distribution (no token has more than 5% probability), the model is genuinely uncertain and a wider set of tokens should stay in play. Top-p applies the same threshold in both cases; min-p adapts to how confident the model actually is.
The combined recipe
A typical production sampling configuration for finance applications: - Temperature: 0.3 (sharp but not deterministic) - Top-p: 0.95 (loose cap, mostly a safety net) - Min-p: 0.05 (tight floor against pathological tokens) - Maximum length: task-dependent (256 tokens for summaries, 2048 for analyses) - Stop sequences: defined per task (e.g., a closing tag marking the end of an answer). These settings produce outputs that are sharp enough to be reliable but flexible enough to avoid degenerate repetition. For finance, where the cost of a hallucinated ticker or a malformed JSON is real money, the sharp settings are essential.
What breaks in production
A surprisingly common configuration in research code is "temperature 1.0, no top-p, no min-p." This means: sample from the model's raw distribution, including the long tail of very low-probability tokens. For chat or creative writing, this is occasionally useful (it produces more diverse outputs). For production finance applications, it is disastrous. Roughly 1 in every few hundred tokens will be a pathological choice - a token from the long tail that derails the entire generation. The resulting output might contain hallucinated tickers, malformed JSON, or text in the wrong language. The fix is mechanical: every production inference path must apply min-p (or at minimum top-p) to clip the long tail. The performance cost is zero; the reliability gain is enormous. This single change is one of the easiest "wins" in deploying LLMs for finance.
Key idea
Temperature scales logits; top-p and min-p clip the tail. Production finance defaults sharp: low temperature, a min-p floor, task-specific stop sequences - reliability over literary diversity.