Chapter 43-47 covered mechanism, alignment, sampling, and the agent loop. None of that tells you how to operate the model when multiple callers, model versions, and deployment environments coexist - and you must upgrade weights without rewriting application code. This chapter introduces the service-layer abstraction - the four-level decoupling that turns a collection of model files into a system you can evolve without rewriting code. It is the unglamorous middleware that separates a research prototype from a deployable platform.
Figure 1. Concept map for this appendix entry: fight, desk translation, what to lock.
Why naming a model is the most consequential decision in your stack
Consider the alternative - application code that hard-codes a model name directly into a function call. That single line entangles three independent decisions: the model family (Qwen vs Llama vs Mistral), the model size, and the exact quantisation - so changing any one of them means hunting down and editing every call site in the codebase.
The four layers: alias, fleet, wrapper, base
The minimum-honest abstraction has four layers. Each layer has exactly one job; each layer can change without disturbing its neighbours. The strictness is the point. Application code is allowed to know only alias names. The alias-to-backend mapping lives in one file (the fleet config). The transport details live behind a small library (the wrapper). The weights live in a separate registry that the wrapper consults. A senior engineer joining the team should be able to learn the names of the seven or eight aliases on day one, write working code on day two, and remain happily ignorant of which precise quantisation backs each alias for years - the abstraction protects the team from having to evolve every time the system does.
The fleet warden: lifecycle management at runtime
The fleet config tells you which model serves an alias. It does not tell you whether that model is currently warm in GPU memory, whether it is healthy, or which model should be evicted if VRAM gets tight. Those are runtime questions. A fleet warden is a small process that answers them. The warden reads the fleet config at startup and on file change, reads the runtime state of the inference engine (what models are loaded, how recently each was used, how much VRAM each consumes), and reads a policy that encodes operator intent - for example, always keep the chat specialist warm, allow at most three models loaded, and evict idle models after thirty minutes.
Promotion mechanics and the rollback contract
Once the four-layer abstraction is in place, promoting a new model becomes a file edit, not a deployment: 1. The candidate trains; an artefact appears on disk. 2. An eval harness scores the candidate against the active model on a held-out pack (Chapter 48). 3. A small script edits the line in the fleet config; the prior backend line is preserved in git history. 4. The next request hits the new model on first call. If anything looks wrong, the operator reverts the config edit and the prior model is restored within minutes. The rollback path must actually be exercised, not just designed - a platform that has never rolled back has never proved that rollback works.
Definition
An alias is a named role - chat-specialist, news-classifier, briefing-narrator - that application code references in place of a specific model identifier. The alias is the contract between the caller and the platform. Behind the alias, the platform is free to swap weights, quantisations, providers, and even hardware without breaking the contract.
Worked picture
A naive policy: "the engine handles eviction; ten seconds of cold-load tax once an hour is fine." On a small system, this might be true. A better policy: "preheat the chat specialist on warden start; never evict it; cap loaded models at three; evict idle background models after thirty minutes; keep the fast classifier warm during market hours and let it go idle overnight." The implementation is twenty lines of Python plus a JSON policy file. The user-visible benefit is that the first chat request of every morning answers in two hundred milliseconds instead of ten seconds. There is no honest argument against the explicit policy; the absence of one is purely a failure to think about the question.
What breaks in production
A common rookie error: the new model is promoted, the old artefact is deleted to save disk space, and the rollback path is "re-train the prior model from scratch." This is not a rollback path; it is a re-creation path that takes hours and may not even reproduce the prior model exactly. The discipline: every promoted model leaves the prior artefact on disk under a versioned name (specialist-v1.bin, specialist-v2.bin) for at least the next promotion cycle. Storage is cheap; recovery time is not. The artefact retention policy is part of the rollback contract, not a separate concern.
The single design principle of a serious LLM stack
Swapping the model that serves a given application surface should be a one-line configuration change, not a code change. Everything in this chapter exists to preserve this property as models, providers, hardware, and corpora evolve underneath you.