Anthropic nearly doubles Sonnet 4.6’s benchmark score with memory prompts

Top News · 155 HN points

Agents do not automatically use what they remember. Anthropic nearly doubled Sonnet 4.6’s CL-Bench score by explicitly prompting it to read its memory and test its theories. Its broader memory design separates logging experience from idle-time “dreaming” that reviews logs, corrects mistakes, and prepares notes for future sessions.

Here is what makes this useful:

• A wiki built from about 350,000 internal documents rose from 13% to 48% of graded criteria across 25 questions as dreaming increased from two to 128 sessions.
• Search plus the growing wiki scored 61%, versus 53% for search alone, and saved about 20,000 tokens per question.
• Merger questions needing cross-document synthesis scored 87% with the wiki, versus 73% with search alone.

One catch: Memory instructions do not transfer cleanly across models. Telling Opus 5 and Opus 4.7 to record only failures reduced much of their memory benefit.

Sources: anthropic.com