Agent Memory Needs a Reader

The gain came from using memory

Anthropic reports that explicitly prompting Claude Sonnet 4.6 to read its memory files and test its theories nearly doubled its CL-Bench score. The intervention addressed an observed behavior: Sonnet 4.6 often did not read files in its memory directory and failed to test its theories about why an answer was wrong.

That makes this a result about memory use, not simply memory availability. In Anthropic’s evaluation, retaining information between sessions did not ensure that the model would consult it or follow through on what it found. The additional instructions targeted both retrieval and investigation.

The distinction matters when interpreting the headline. Anthropic’s research post evaluates components of agent memory, including memory notes, reflection over transcripts, and wiki-like representations.

The reported near doubling also has a specific scope: Sonnet 4.6, CL-Bench, and instructions to read memory and explicitly test theories. It is not a reported doubling across every memory-enabled workflow.

Anthropic identified a gap between having memory available and using it. A memory directory can contain relevant experience while the agent still skips the steps needed to benefit from it. In this experiment, the prompt was part of the memory system’s effective behavior, not merely a description of the task.

Logging and dreaming do different jobs

Anthropic defines agent memory as knowledge gained from experience and retained between sessions in context the model can read later, such as files.

Its broader design separates recording experience from reviewing that experience. Logs or transcripts preserve what happened. Background processing, which Anthropic calls “dreaming,” can reflect on those records, correct mistakes, and produce notes or other representations for future sessions.

That gives the memory system more than one opportunity to go wrong. The stored record may need correction. The representation prepared from it may be useful or unhelpful. The model may then fail to read or act on that representation, as Anthropic observed with Sonnet 4.6.

The distinction between logging and dreaming is therefore substantive. A transcript records an interaction. Dreaming can turn that interaction into something intended for later use, rather than leaving every future session to work directly from the original record.

Anthropic’s evaluations cover several representations rather than presenting one universal memory format. Memory notes, transcript reflection, and a knowledge wiki are all part of the investigation. The wiki experiment, in particular, tests whether repeated background work can prepare information that helps with later questions.

The reported design also makes maintenance explicit. Anthropic describes dreaming as reviewing logs to fix mistakes or produce new notes and representations for later sessions.

The loaded index is not the whole memory

Anthropic gives a concrete example of selective loading in Claude Code’s auto memory. According to the post, Claude loads only the first 200 lines or 25 KB of MEMORY.md, not all of its auto memory.

This separates what is immediately visible from what remains available to read. The loaded material is an entry point into memory, not the complete body of retained experience.

Read alongside the Sonnet 4.6 result, that distinction is important. Anthropic observed that the model sometimes did not read files from its memory directory. Having an index and having additional files available are not the same as successfully consulting those files. The reported prompt intervention directly addressed that missing behavior.

Preloading everything is not presented as a costless alternative. Anthropic notes that injected memories consume context tokens. Incorrect notes can also be carried into every session, repeatedly exposing the model to the same bad information.

Nor does loading a note guarantee compliance with it. Anthropic explicitly identifies following memory as a separate concern. The practical boundary in these experiments is therefore between retained information, information actually read, and information acted upon. The near-doubling result came from instructions that reached beyond retention into those latter behaviors.

The wiki results depend on the task

Anthropic’s internal-document experiment provides a different view of the system: whether background work can build a useful knowledge representation from a large corpus. The wiki was built from about 350,000 internal documents and evaluated on 25 questions.

As wiki-building increased from two dreaming sessions to 128, Anthropic reports that wiki-only performance rose from 13% to 48% of graded criteria. Those percentages describe satisfaction of graded criteria, not a reported fraction of questions answered completely correctly.

The combined search-and-wiki result was stronger than the wiki alone. Anthropic reports an aggregate score of 61% for search plus the wiki. Search alone scored 53%, while the final wiki-only score was 48%.

Those results support a narrower reading than replacing search with a wiki:

• Search alone scored 53% on the aggregate evaluation.
• The wiki alone reached 48% after 128 dreaming sessions.
• Search plus the wiki reached 61%.

In this evaluation, the strongest aggregate result came from combining the two resources. The wiki did not outperform search by itself on the aggregate score.

The task-level result was more favorable to the wiki for questions about mergers that required cross-document synthesis. Anthropic reports 87% when Claude consulted the wiki, compared with 73% using search alone. That is a result for those synthesis-heavy items, not the aggregate result for all 25 questions.

Anthropic also reports that combining the wiki with search saved about 20,000 tokens per question for this corpus. The experiment therefore offers two distinct reported benefits: better graded performance and lower per-question token use. Neither should be detached from the corpus and evaluation in which it was measured.

Memory instructions are model-specific

Anthropic’s observations do not support treating the Sonnet 4.6 prompt as a model-independent fix. For Sonnet 4.6, the instructions addressed observed gaps in retrieval and follow-through.

The reported results also show sensitivity to what the agent is told to retain. Telling Opus 5 and Opus 4.7 to record only failures reduced much of their memory benefit. A rule that narrows memory to unsuccessful experiences was therefore not uniformly beneficial in those evaluations.

This is distinct from the Sonnet 4.6 intervention. One instruction changes what gets written. The other changes whether the model reads memory and tests its explanations. Both affect the memory system, but at different points.

For an engineering evaluation, those are separate behaviors to inspect: what the model records, whether it retrieves that material, and whether it follows through. Anthropic’s results give examples of sensitivity at each of those points. They do not establish one recording policy or one retrieval prompt as the best choice across the models discussed.

The savings come with maintenance work

Anthropic reports token savings for answering questions with search plus the wiki, while its scaling experiment involved increasing dreaming from two sessions to 128.

The per-question savings should therefore be read as a query-time result, alongside the background work that produced the wiki. They do not, by themselves, establish the overall cost of the memory system.

Anthropic also notes that the wiki was less useful in other cases, reinforcing the task dependence visible in the internal-document results.

Correctness remains a maintenance concern as well. Memory can become incorrect or stale. Dreaming can help fix mistakes, but preloading an incorrect note can repeatedly inject the same error into later sessions. Retaining more material does not remove that risk, and loading it does not ensure the model follows it appropriately.

The work examines agents that carry experience across sessions and a wiki built from an internal-document collection. Anthropic supplies concrete behaviors to evaluate: whether memory is read, whether theories are tested, whether background organization helps, and whether combining a wiki with search improves the target task.

The central result is not that persistence alone improves an agent. Anthropic’s experiments show benefits from particular combinations of retained experience, background organization, retrieval, and explicit follow-through. In the Sonnet 4.6 evaluation, memory files were available, but Anthropic observed inconsistent retrieval and follow-through until it explicitly prompted those behaviors.

Try it

Start with the dreaming and knowledge-wiki cookbooks linked from Anthropic’s research post at https://www.anthropic.com/research/components-of-agent-memory. The supplied source provides no runnable commands or access requirements, so there are no command steps to reproduce here.

Sources

• https://www.anthropic.com/research/components-of-agent-memory