The Agent Stack

The part of an agent you actually look at: where you type a request and watch the work come back. It handles presentation rather than execution, so the same agent can appear as a terminal, an editor panel, a native app, or a chat bot. The Agent Client Protocol calls this role the client.

Everything the model gets to see before it answers. Most of it is pulled off your file system — AGENTS.md, skills, source files, and the output of earlier tool calls. This layer is where context comes from, not the context window itself; the harness decides what to load and in what order.

The code wrapped around the model that turns it into an agent. It runs one loop: read context, call the model, evaluate what came back, run the tools it asked for, then feed the results in and go again until the work is done. Claude Code, Codex, and Nori are each a harness.

What the agent can actually do to the world outside the model. Run a shell command, read and edit files, fetch a URL, call an MCP server. Each harness defines its own set and its own instructions for using them, so the same model behaves differently depending on which tools it was handed.

The line where your control stops. Past this HTTP POST you cannot see or change how the request is served. Two things you can still set at the boundary: enterprise plan policy applied across a whole team, and an AI gateway — a stand-in API that meters spend per user, restricts which models are reachable, and issues one key for many providers.

The data center that runs the model. Your request lands in a provider region, is routed to a machine, and is served by two neighboring services: a cache and the inference service. The GPUs sit inside inference, and the model weights are loaded onto them.

Reusing the work of your last request instead of paying for it twice. Providers cache a prompt's prefix, so anything that changes early — a system prompt, a tool definition — invalidates everything after it. Those caches usually live five minutes to an hour, which is why harness choice and long gaps between turns show up directly on your bill.

The service that turns your tokens into the model's tokens. It reads the whole prompt in one pass, then generates one token at a time, batching your request alongside others to keep the hardware busy. A KV cache holds the intermediate state so it does not re-read the whole conversation for every token.

The chip that does the arithmetic. A data center GPU packs a few hundred small processors, each holding tensor cores that multiply a tile of a matrix per instruction, alongside high-bandwidth memory. Generation speed is limited by how fast weights move out of that memory, not by raw math.

The learned numbers that are the model. A token enters at one end and passes through layer after layer of matrix multiplies until a single output token falls out the other. Many large models are sparse: in a mixture of experts, a router sends each token to only a fraction of the network, so capacity can grow much faster than the cost of running it.