How Deep Agents Manages Context: Offloading and Summarization

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: How Deep Agents Manages Context: Offloading and Summarization · Channel: LangChain · 04:45 · Watch on YouTube

Topics: AI agents, Prompt and context engineering, AI developer tools

Summary

Agents can experience “context rot” as long tasks fill the model’s context window with instructions, tool results, and other information, which can hurt reasoning and accuracy. Deep Agents uses context engineering techniques to keep unnecessary information out of context and manage what is already there. Offloading saves large tool results and human messages so the full content remains recoverable while the model receives a preview. Summarization condenses older context when token usage crosses a configurable threshold, preserving the original messages in conversation history. Together with skills, subagents, and long-term memory, these techniques help provide relevant context when needed.

Key points

Context rot describes how accumulated history, noise, and tool output can worsen an agent’s reasoning and accuracy.
An agent consists of a model and a harness that supplies the right context at the right time.
Context engineering can prevent unnecessary information from entering the context window or manage information already inside it.
Deep Agents includes offloading, summarization, skills, subagents, and long-term memory.
Offloading saves large tool results in a virtual file system and gives the model a preview; the full result can be retrieved with the read file tool.
Long human messages, such as a visa document or transcript, can also be offloaded; the stated threshold is 50,000 tokens.
Summarization checks before each model call and, by default, is triggered at an 85% context-usage threshold.
Messages above the keep cutoff line are summarized and saved in conversation history; messages below it are appended verbatim to the summary.
Offloading and summarization work together: offloading intercepts inputs before they enter context, while summarization acts on content already there.

Chapters

Start	Chapter	Summary
00:01	Context rot and context engineering	The video introduces context rot and explains how an agent’s harness manages context through prevention and filtering.
01:03	Offloading large inputs	Deep Agents offloads large tool results and human messages, keeping the full content recoverable while sending the model a preview.
02:37	Summarizing accumulated context	Summarization condenses older messages when context usage reaches a configurable threshold, while keeping the original messages in conversation history.
04:13	How offloading and summarization work together	The video recaps how the two techniques complement each other and mentions other Deep Agents context techniques.

Quotes

We call this context rot. — 00:01

the job of the harness is to get the model the right context at the right time. — 00:32

The model sees just the first and last few lines — 01:35

Summarization can be used to condense our agent’s entire context window into a compacted version of what has happened so far — 02:37

offloading intercepts inputs before they enter context, whereas summarization acts on content already in the context window. — 04:13

Concepts and entities

context rot, context engineering, offloading, summarization, Deep Agents, LangChain, LangChain Academy

Full transcript (original language)

00:01 If you’ve ever crammed for an exam the night before, then you know the feeling. The more you study, the more the earlier chapters start to slip. Well, agents have the same problem. As they take on longer tasks, they take on more and more information like instructions and tool results. And eventually, this becomes far too much context to comfortably fit in a model’s context window. And the more history, noise, and tool output a model carries, the worse its reasoning and accuracy tend to get. We call this context rot. So, what’s going on here? To simplify, an agent is just a model

00:32 plus a harness. And the job of the harness is to get the model the right context at the right time. The context engineering techniques at the agent’s disposal to address context rot fall primarily into two categories. The first of which is preventing unnecessary information from even entering the context window in the first place. And the second is about managing and filtering what’s already in the context window. It’s the harness that applies these methods strategically to get the model the right context at the right time for the given task. Now, wiring up

01:03 a harness that does this well is easier said than done and a lot of work. The Deep Agents harness offers these context engineering techniques out of the box with offloading, summarization, skills, subagents, and long-term memory. To recap, the first line of defense is to prevent unnecessary information from even entering the context window in the first place. be selective in this retrieval process. The second is to manage and filter what’s already in the context window to control what gets constantly sent to the model. Offloading is an example of

01:35 intercepting inputs before they enter context. Say we’re chatting with our agent and we ask it to query something. It calls a tool to do that, query_db, and we can see that the tool result from querying this database is huge, over 20,000 tokens. The tool result is then sent to a virtual file system and saved in a folder called large_tool_results. This means the full answer is always recoverable, but we do not send everything to the model. The model sees just the first and last few lines and can page through the full result with

02:06 the read file tool if it needs to. Offloading works with more than just tool results, though. It works with human messages, too, like a long visa document or transcript. And the threshold here is slightly higher at 50,000 tokens. This is saved to conversation_history instead of large tool results, but it’s the same principle. The model gets a preview. And this is how we try to keep as much unnecessary information out of the context window as possible. And it’s the same process.

02:37 However, despite our best attempts to offload verbose tool calls, context will nonetheless begin to build up over a long run. This is where summarization steps in. Summarization can be used to condense our agent’s entire context window into a compacted version of what has happened so far to basically give it space to keep working on other things. Here’s our context window. And this 85% threshold is just the default value. You can adjust this so that summarization kicks in at a lower or higher token usage threshold. A check runs right before every model call to see if

03:08 summarization is necessary. As our messages pile up, you can see the token usage bar on the right increasing. Now, the reason it actually overshoots 85% here is because the check runs right before each model call and a turn with several tool calls can add a lot of context in between model calls. Now, everything above the keep cutoff line gets summarized and you can see that the token usage bar went way down. Those messages are not lost though. Similar to offloading, they

03:38 are saved in conversation history and can be accessed if needed. But now only a summary is sent to the model. Then the messages from below the keep cutoff line are appended verbatim to the summary and this loops. So now you can see that our context window has been summarized once and we keep chatting with our agent and information accumulates again until the process repeats. This time the summary includes the previous summary and it keeps looping.

04:13 To recap, offloading intercepts inputs before they enter context, whereas summarization acts on content already in the context window. So token usage in context is growing and the small bump is offloading. A large result came in when its preview landed in context and summarization reduces the inevitable gradual buildup. These two work together. Now, offloading and summarization aren’t the only context engineering techniques that come out of the box with Deep Agents. Skills, subagents, and long-term memory all work to

04:45 keep context out of the context window until it’s absolutely necessary and relevant. If you’re interested in learning more, I highly recommend you check out our free online self-paced course on LangChain Academy.