# AI Agents Do Not Just Need More Context. They Need a Context Budget A long context window sounds like an obvious advantage for an AI agent. The agent can retain more search results, tool outputs, intermediate reasoning, and evidence. Give it enough context, and perhaps it will stop forgetting what it discovered ten steps ago. But storing more information is not the same as managing it well. The **ContextBudget** paper argues that long-running agents face a more specific problem: they must decide what to preserve, what to compress, and when to do so under a finite context budget. Most context-compression approaches, according to the paper, do not explicitly condition those decisions on how much capacity remains. The authors call this a **budget-free formulation**. That distinction matters because the wrong compression policy can fail in opposite directions. ## Why an agent’s context keeps growing Many language-model agents alternate between reasoning and acting. They decide on an action, call a search engine or another tool, inspect the result, update their reasoning, and repeat. ReAct, an influential early agent framework, formalized this interleaving of reasoning traces and environment actions. Each additional step can contribute another observation and another piece of reasoning to the interaction history. A simplified trajectory might look like this: ```text Question → Search query → Search results → Reasoning → Another query → More results → Revised reasoning → Another tool call → More observations ``` This is useful information, but it is not free. The history grows as the task continues. Eventually, the agent either approaches the model’s context limit or carries so much material that identifying the relevant evidence becomes difficult. Research on long-context language models has already shown that nominal context capacity does not guarantee reliable use of all the information inside it. In *Lost in the Middle*, Liu and colleagues found that performance could fall substantially when relevant information appeared in the middle of a long input rather than near its beginning or end. The practical problem is therefore not only: > Can the information fit? It is also: > Can the agent retain the right information in a form it can still use? ## Compression helps but introduces another decision One response is to compress the interaction history. Instead of repeatedly sending every raw search result and reasoning trace back to the model, the system can summarize older material into a smaller representation. MEM1, for example, trains agents to update a compact internal state that combines previous memory with new observations while discarding information judged irrelevant or redundant. Compression can extend an agent’s effective working horizon. It can also reduce repeated processing of material that no longer affects the task. But compression is lossy. A summary may preserve a conclusion while dropping the qualification that made it reliable. It may retain an entity but lose the source from which it came. Two observations that initially look redundant may later become important because they disagree. Once that detail has been removed, a later reasoning step cannot recover it from the compressed history alone. This creates a trade-off: ```text More retained detail ↕ Less context capacity ``` The compression mechanism must manage both sides. Simply compressing more often is not necessarily safer. ![Comparison of over-compression under a relaxed context budget and overflow under a tight budget.](https://cdn.hashnode.com/uploads/covers/63a80bd9b1011ed9ec6d049c/9284d53f-953d-48bf-bf99-b2d1383c8544.png align="center") ## Failure mode one: compressing too much Imagine an agent has a context allowance of 16,000 tokens but applies the same aggressive summarization policy designed for a 4,000-token limit. The agent may remain comfortably within capacity. It may also discard evidence that it had enough room to preserve. ContextBudget describes this as **over-compression under relaxed budgets**. A budget-free method can remove information unnecessarily because its behavior does not adapt to the capacity currently available. This is especially risky in research and multi-step search tasks. An observation that seems unimportant at step five may become the missing connection at step fifteen. Compression is not merely a storage operation. It changes the information on which future decisions will depend. ## Failure mode two: compressing too little The opposite problem appears when the budget is tight. A system may postpone summarization until its context reaches a fixed threshold. By then, an incoming tool result may push the trajectory beyond the permitted window. The system must then truncate something, trigger emergency compression, or continue with an incomplete history. ContextBudget describes this as **under-compression under tight budgets**. The resulting overflow can cause truncation or brittle reasoning failures. The important word is *tight*. The same amount of retained history may be harmless under one deployment limit and unsafe under another. A static compression rule cannot respond intelligently unless the remaining capacity is part of the decision. ## Why “budget-free” is the real weakness A compression policy answers at least three questions: 1. **When should compression happen?** 2. **How much of the history should be compressed?** 3. **Which information should survive?** A budget-free method can answer these questions using fixed schedules, thresholds, or learned representations. What it lacks is an explicit signal connecting those decisions to the context capacity available at that moment. That can make the policy too aggressive when memory is plentiful and too conservative when memory is scarce. ContextBudget reframes the problem as a **budget-constrained sequential decision process**. Instead of treating summarization as a fixed maintenance operation, the proposed framework lets the agent adapt its compression decisions to its remaining context capacity throughout the trajectory. The paper reports that this budget-aware approach was more stable than the evaluated alternatives as maximum context length decreased from 16,000 to 4,000 tokens. In the authors’ experiments, fixed or budget-free approaches were more likely to lose information through repeated compression or delay compression until the context was saturated. These are results from the paper’s compositional question-answering and web-browsing evaluations—not evidence that the method will behave identically in every production agent. That limitation is important. **ContextBudget is an April 2026 arXiv preprint under review**, and its production behavior, latency costs, and generality beyond the evaluated tasks remain open questions. ## The engineering lesson The useful takeaway is not that every agent should summarize its history. It is that context management should respond to actual resource pressure. When capacity is sufficient, preserving detail may be the better decision. As the available budget shrinks, selective compression becomes more valuable. Under severe constraints, aggressive aggregation may become necessary. In other words: > An agent should not forget on a fixed schedule. It should forget according to the evidence it needs and the capacity it has left. A larger context window can delay the problem. It does not remove the need to make that decision. * * * ## References * Wu, Y., Zheng, Y., Xu, T., et al. [*ContextBudget: Budget-Aware Context Management for Long-Horizon Search Agents*. arXiv preprint, submitted April 2, 2026.](https://arxiv.org/html/2604.01664v1) * Liu, N. F., Lin, K., Hewitt, J., et al. [*Lost in the Middle: How Language Models Use Long Contexts*.](https://arxiv.org/abs/2307.03172) * Yao, S., Zhao, J., Yu, D., et al. [*ReAct: Synergizing Reasoning and Acting in Language Models*.](https://arxiv.org/abs/2210.03629) * Zhou, Y., et al. [*MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents*.](https://arxiv.org/abs/2506.15841)