Stop Buying Bigger Context Windows. Compaction Beats Them
A widely circulated paper this quarter tested a simple comparison: give an agent a large raw context window, or give it a quarter of that window plus a learned compaction step that summarises and discards its own history. The compacted agent won on long-horizon tasks, consistently, across model families.
This should not be surprising, and yet almost every product decision in the industry over the past eighteen months has gone the other way.
Why Bigger Windows Underperform
Three reasons, all mundane.
Attention dilutes. The model has a fixed budget of attention to distribute. Spread across two million tokens, most of which are irrelevant, less of it lands on the tokens that matter. This is the "lost in the middle" effect and it is not solved, only mitigated.
Stale context misleads. In a long agent run, early context contains decisions that were later reversed, hypotheses that turned out wrong and tool outputs that have since changed. Keeping all of it means the model is reasoning over a mixture of current and obsolete state, and it has no reliable way to tell which is which.
Cost scales linearly, value does not. Every token in the window is billed on every turn. A 500K context agent taking fifty turns pays for 25 million input tokens even if only 30K of context was ever relevant.
What Compaction Looks Like in Practice
At its simplest: when context crosses a threshold, ask the model to write a structured summary of the run so far, then start a fresh context containing the summary, the original goal and the current working state. Discard the raw history.
The details that make it work well:
- Summarise into a schema, not into prose. Goal, decisions made, decisions reversed, files touched, current blocker, next step. Free-form summaries lose exactly the structured details you need.
- Keep the last two turns raw. The immediate working state should not be summarised, because that is the part the model is actively reasoning over.
- Compact on a token threshold, not a turn count. A run with three enormous tool outputs needs compaction sooner than a run with thirty small ones.
- Never summarise the original instruction. Goal drift across compactions is the single most common failure mode, and it is entirely preventable by carrying the original text through verbatim.
The Cost Case
Take an agent averaging 40 turns with a 200K context. Without compaction that is roughly 8 million input tokens per run. With compaction holding context at an average 60K, it is roughly 2.4 million. On Claude Opus 5 at $5 per million that is $40 versus $12 per run.
At a hundred runs a day the difference is around $1,000 daily. Model your own workload on the cost calculator and the token counter.
Where the Vendors Are
Anthropic's compaction layer in Opus 5 is the most developed implementation currently shipping, and the reason the day-long-run claim holds up. Others are building similar mechanisms. If you are on a framework like LangGraph you can implement checkpointed compaction yourself today.
The tools in our Hermes-OpenClaw directory vary widely in how well they handle this. It is worth checking before you commit to one for long-running work.
The Uncomfortable Implication
Context window size is a marketing number. It is easy to measure, easy to compare and only loosely related to whether an agent completes a four-hour task. Teams that optimised for window size over the past year mostly bought expensive context they did not use well.
The variable that actually predicts long-horizon success is how good the system is at deciding what to forget.