Context Window

The context window is all the text a language model can reference while generating a response, including the response it is currently producing (Context windows — Claude Docs). It acts as the model's working memory for a single inference pass — distinct from the training corpus baked into its weights — and it starts empty on every request: whatever the model needs to "know" this turn must physically occupy window space. For teams running AI agents, the window is the budget that every system prompt, tool definition, tool result, and reasoning trace draws from.

Why it matters

Agents spend context far faster than chat does. Each tool round-trip appends both the call and its full result to the message history that the next inference pass must carry (Context windows — Claude Docs), so an agent harness is, to a first approximation, a machine for deciding what deserves window space — which tools to expose, how verbose their outputs are, and when to summarize or delegate.

Overflow is an explicit failure mode to design for, not an edge case. On the Claude API, if the input alone exceeds the window, the request fails with a 400 invalid_request_error ("prompt is too long") on every model; on Claude 4.5 models and newer, a request whose input plus max_tokens overshoots the window is accepted, but generation stops with stop_reason: "model_context_window_exceeded" if the ceiling is actually hit (Context windows — Claude Docs). Retry, truncation, and compaction paths in a harness need to handle both shapes.

Capacity also does not buy reliability. Anthropic's own product docs state that "as token count grows, accuracy and recall degrade, a phenomenon known as context rot" (Context windows — Claude Docs) — curating what enters the window matters as much as raising its size (see Boundaries and misconceptions below). Cost and latency are keyed to window use in vendor-specific ways: OpenAI prices GPT-5.5 prompts above 272k input tokens at 2x the input rate and 1.5x the output rate for the full session (GPT-5.5 — OpenAI Docs, as of 2026-07-10), Anthropic bills long-context requests on its 1M-window models at standard pricing (Context windows — Claude Docs, as of 2026-07-10), and Google's long-context guide notes that longer queries generally raise time-to-first-token (Long context — Gemini API Docs).

How it works

Everything in the request counts. On the Claude API the window holds the system prompt, every message — including tool results, images, and documents — the tool definitions, and the output the model generates for the turn, extended-thinking tokens included (Context windows — Claude Docs). Prompt caching does not free space: cached input is reported under separate usage counters (cache_read_input_tokens, cache_creation_input_tokens), but cached prefixes "still occupy the context window" — caching changes what you pay for those tokens, not whether they count (Context windows — Claude Docs).

Tokens, not words — and the ratio moves. Windows are measured in tokens, and the token-per-word ratio is not stable even within one vendor. Anthropic's model docs put 1M tokens at roughly 555k words on the tokenizer introduced with Claude Opus 4.7, versus roughly 750k words on the earlier tokenizer — the same text produces roughly 30% more tokens on the newer one, with the exact increase depending on content (Models overview — Claude Docs, as of 2026-07-10). Token budgets and max_tokens settings tuned on one model do not transfer to another.

Current sizes. As of 2026-07-10, the 1M-token class spans multiple vendors:

Model	Context window	Max output (sync API)	Source
Claude Fable 5, Claude Opus 4.8, Claude Sonnet 5	1M tokens	128k tokens	Claude Docs
Claude Opus 4.7, Opus 4.6, Sonnet 4.6	1M tokens	128k tokens	Claude Docs
Claude Haiku 4.5, Sonnet 4.5, Opus 4.5	200k tokens	64k tokens	Claude Docs
GPT-5.5 (OpenAI)	1.05M tokens	128k tokens	OpenAI Docs

On every Claude model with a 1M window, 1M is the default — no beta header needed (Context windows — Claude Docs, as of 2026-07-10). On the Message Batches API, Claude Opus 4.8, Opus 4.7, Opus 4.6, Sonnet 5, and Sonnet 4.6 raise max output to 300k tokens via a beta header (Models overview — Claude Docs, as of 2026-07-10). Google's Gemini 2.5 Pro shipped with "a 1 million token context window (2 million coming soon)" at its March 2025 launch (Google Blog, 2025-03-25); Google's current docs describe Gemini models as having windows "of 1 million or more tokens" (Long context — Gemini API Docs, as of 2026-07-10).

Why capacity is architecturally expensive. In transformer self-attention every token can attend to every other token, so n tokens create n² pairwise relationships — as context grows, the model's ability to capture those relationships "gets stretched thin." Models also develop attention patterns from training data dominated by shorter sequences, leaving them "fewer specialized parameters" for context-wide dependencies; the resulting degradation is "a performance gradient rather than a hard cliff" (Effective context engineering — Anthropic). On the serving side, generation relies on the kv-cache: each token's attention key/value vectors are stored so they are not recomputed for every new token — without the cache, sampling would be quadratic in sequence length — and per-token cache memory scales with the model's layer count, head count, and head dimension (Transformer Inference Arithmetic — kipply). Accelerator memory for that cache is a hard constraint on how much context a deployment can serve at a given batch size; that analysis is the 2022 architectural baseline, and modern serving stacks add further optimizations on top of it.

Boundaries and misconceptions

The window is not the model's knowledge. Training data shapes the weights; the context window is a per-request buffer the vendor's docs explicitly contrast with "the large corpus of data the language model was trained on" (Context windows — Claude Docs). A fact that is in neither the weights nor the window does not exist for the model this turn — retrieval, tool calls, and memory files are all mechanisms for moving facts into the window.

A bigger window is not better recall. Chroma's "Context Rot" technical report (July 14, 2025; Hong, Troynikov, Huber) evaluated 18 models across the Claude, GPT/o-series, Gemini, and Qwen lineups and found their performance "grows increasingly unreliable as input length grows" — including on simple retrieval and repeated-word copy tasks (Context Rot — Chroma). The phenomenon now has a name in vendor docs themselves (see the Anthropic quote above) and its own page: context-rot.

Position inside the window matters — but the effect is task-dependent. "Lost in the Middle" (Liu et al., 2023) found on multi-document QA and key-value retrieval that "performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models" (arXiv:2307.03172). Chroma's 2025 suite complicates the picture: its standard single-needle test showed no notable variation by needle position, while its repeated-words task scored highest when the target appeared near the beginning of the sequence — an effect that strengthened as input length grew (Context Rot — Chroma).

Coherent surroundings can hurt retrieval. Counterintuitively, Chroma found models "perform better on shuffled haystacks than on logically structured ones" — a pattern consistent across all 18 models: orderly narrative flow around an embedded fact made it harder, not easier, to retrieve (Context Rot — Chroma).

Vendor recall numbers describe a best case. The standard measurement is the needle-in-a-haystack test, created by Greg Kamradt with first published runs in November 2023: plant one out-of-place fact ("needle") in a long document, sweep needle depth from 0–100% across increasing context lengths, and score retrieval as a depth-by-length grid (LLMTest_NeedleInAHaystack — GitHub). Google's long-context guide reports single-fact needle retrieval at "up to 99% accuracy in many cases," while cautioning that with multiple needles "the model does not perform with the same accuracy" and that "performance can vary to a wide degree depending on the context" (Long context — Gemini API Docs, as of 2026-07-10). Chroma likewise caveats that its evaluation "is not exhaustive of real-world use cases" — production long-context work needs synthesis and multi-step reasoning, not isolated lookup — and states plainly that it does "not explain the mechanisms behind this performance degradation" (Context Rot — Chroma).

Adjacent terms that get conflated with the window itself:

Term	Layer
Context window	Raw per-request capacity (this page)
Context rot	Reliability degradation as that capacity fills
Context compaction	Mitigation: summarize history to keep working near the limit
KV cache	Serving-side memory that physically backs the window
Prompt caching	Billing/latency optimization; frees no window space

In practice

At the API level, vendors increasingly expose the window as a managed resource rather than a silent limit. On the Claude API, Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5 have "context awareness": the API injects the model's total token budget into the system prompt (a budget:token_budget tag) and, after each tool call, a running usage update, so the model can pace long tasks against remaining space instead of guessing; image tokens are included in these budgets (Context windows — Claude Docs). Newer models — Claude Opus 4.7 and later, and Fable 5 — do not receive the injected tags; an explicit budget is set through a separate task-budgets beta instead (Context windows — Claude Docs, as of 2026-07-10). Server-side compaction (in beta on current Claude models: Opus 4.6 and later, Sonnet 4.6 and later, and Fable 5) summarizes earlier conversation on the server so a session can continue past the raw limit, and context editing offers narrower levers — clearing old tool results or thinking blocks (Context windows — Claude Docs).

Claude Code shows what this looks like inside an agent harness. Its window holds conversation history, file contents, command outputs, CLAUDE.md, auto memory, loaded skills, and system instructions; as it fills, the harness clears older tool outputs first, then summarizes the conversation — preserving the user's requests and key code snippets while detailed early instructions may be lost, which is why the vendor's guidance is to put persistent rules in CLAUDE.md rather than rely on history surviving compaction (How Claude Code works — Claude Code Docs). If a single oversized file or tool output refills context immediately after each summary, Claude Code detects the thrashing and stops auto-compacting after a few attempts, surfacing an error instead of looping (How Claude Code works — Claude Code Docs). A 1M window is available in Claude Code for Fable 5, Sonnet 5, Opus 4.6 and later, and Sonnet 4.6, selected as a [1m] model variant — except Sonnet 5, which runs at 1M with no variant to select (Explore the context window — Claude Code Docs, as of 2026-07-10).

Isolation is the other big lever: Claude Code subagents run in their own separate context window — the parent delegates a task, the subagent does all its file reads and searches in its own window, and only its final text response plus a small metadata trailer (token counts, duration) returns to the parent's context (Explore the context window — Claude Code Docs). Skill descriptions are listed at session start, but full skill bodies load only when a skill is actually invoked (Explore the context window — Claude Code Docs).

These harness behaviors instantiate the general techniques Anthropic recommends for agents: just-in-time retrieval (keep lightweight identifiers like file paths, stored queries, and links; load data via tools at runtime), structured note-taking outside the window (an agent playing Pokémon kept precise step and level tallies by persisting and reloading notes), compaction — which "typically serves as the first lever" for long-horizon coherence — and sub-agent architectures where each worker starts with a clean window and returns a condensed summary, often 1,000–2,000 tokens (Effective context engineering — Anthropic).