Context windows now run to millions of tokens, yet reliability decays as the window fills, so inside a single run, the capacity you paid for depreciates the more you use it.
The same model that scores near-perfect on a needle-in-a-haystack test at a million tokens will quietly misread a file it loaded earlier in the same run.
That gap between what a context window can hold and what the model reliably uses once it is full is the most under-budgeted cost in agent engineering right now.
The window is sold as storage; in practice, it is a depreciating asset, and the depreciation occurs within a single run before you ever reach the token limit.
What context rot actually is
Context rot is the measured decline in a model’s reliability on the contents of its own context window as that window fills not at the token limit, but well before it.
Chroma’s July 2025 technical report tested 18 leading models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found that none of them use their context uniformly.
Accuracy slips as input grows, even for trivial tasks like retrieval and repeating a list of words.
The mechanism predates the agent era.
“Lost in the Middle” (Liu et al., 2023) documented a U-shaped curve: models attend best to the start and end of a context and worst to the middle, so the more you load, the more of it lands in the low-attention zone.
The window depreciates as you fill it
A context window costs more per token and returns less per token as it fills, which is the precise signature of a depreciating asset.
Two costs move in the wrong direction at once.
The quality cost is the rot itself: marginal tokens dilute the signal the model needs and push relevant facts into its blind spots.
The compute cost is structural: self-attention scales with the square of the sequence length, so doubling the context roughly quadruples the attention work required to process it, while the key-value cache that holds the running conversation grows linearly in memory at every step.
Prompt caching softens the cost of re-sending a long prefix, but it buys nothing in terms of comprehension.
A cached token is still a token the model must attend over, and caching the history does not make the model read it any better.
You pay more in tokens and in compute for context that is worth less.
The evidence is sharpest in coding agents
SlopCodeBench, a 2026 benchmark that requires an agent to keep building on its own code across checkpoints, found no agent to solve a single problem end-to-end, with the best agent passing only 14.8% of checkpoints.
Structural erosion increased in 77% of trajectories across the 15 agents evaluated, and verbosity in 75.5% of the code did not merely stop improving; it decayed as the run progressed.
The result that should worry anyone gating agents into production is about slope, not level.
Prompt-side fixes moved the starting quality but not the rate of decay degradation resumed at the same rate regardless of how the run began.
Pass-rate benchmarks miss this entirely, because a patch that passes its tests can still be the one quietly eroding the codebase.
David Parnas named the underlying phenomenon “software aging” in 1994.
The new wrinkle is that aging now occurs within a single session, driven by the model rereading and rewriting its own output.
Bigger windows are not the fix, and the labs know it
The labs shipping million-token windows do not market the window as the solution; they ship context-management tooling because the window alone does not hold quality.
Anthropic’s own guidance treats context as a finite resource with diminishing returns and warns that windows of every size are subject to context pollution, not only small ones.
Its compaction feature exists for a reason the docs state outright: a longer history is one the model attends to less reliably, so compaction replaces old turns with a summary and continues from there rather than from the raw transcript.
The distinction worth keeping has two halves.
Harness depreciation is scaffolding going stale across model generations: Anthropic found a context-reset workaround for one model’s premature wrap-up that became dead weight on the next, stronger model.
Context rot is the within-run sibling: the same long context a harness must manage today, decaying token by token as the single run proceeds.
Engineering for a budget you spend down
Treat the context window as a budget you spend down within a run, not a capacity you fill; the cheapest and most reliable token is the one you keep entirely out of the window.
The patterns that work all push the same direction.
Just-in-time context retrieving the few files or facts a step needs, instead of loading the repository up front, keeps the window small and the signal high.
Compaction, tool-result clearing, and memory moves stale tokens out of active context and durable state into files, so a lost summary is recoverable from git history or a progress note rather than being catastrophic.
Tool curation belongs in the same budget.
Scoping an agent to the smallest set of tool definitions that covers its task attacks the same token tax that connecting every server quietly imposes.
The business translation is uncomfortable but clarifying.
Every token you put in the window is billed on the input side, processed at quadratic cost, and, past a point you rarely measure, paid for again in reliability you did not know you were spending.
What is now at stake
The capacity question is settled: the windows are large and getting larger.
The open question is operational, and nobody has a clean answer to it yet.
How do you price a resource that costs more and works less the more you use it, when you cannot fully watch it in real time?
Thanks for taking the time to read my article!
Comments
Loading comments…