Agent memory: what to keep, what to forget and how to avoid repeating the error

With memory, an agent went from close to zero to 70% accuracy and started solving tasks in around five steps, instead of twenty.

The same study recorded the other side. The agents started citing earlier runs that were wrong, and reusing those results with even more confidence than before.

Memory had given the error the look of precedent.

Both results come from the same paper, and that is what makes the subject interesting. Memory is no feature you switch on. It is a system with five operations, and most teams build two.

A complete memory system does five things: it stores, retrieves, updates, compresses and forgets.

Store and retrieve are the two any tutorial teaches, and the two any library delivers. The other three require deciding uncomfortable things: when a fact turned into a lie, what fits in a summary and what deserves deletion.

This post covers the three with the least coverage above all. The distinction between episodic and semantic memory, and the consolidation process between them, is already well covered in Portuguese by other authors and needs no retelling here.

Measuring the gain before discussing the risks pays off, because it is large.

In "Memory Scaling for AI Agents", published on 10 April 2026 by Wenhao Zhan, Veronica Lyu, Jialu Liu, Michael Bendersky, Matei Zaharia and Xing Chen, the Databricks team measured the effect of giving an agent memory (Databricks, accessed 28 August 2026).

Accuracy with labeled data went from close to zero to 70%, beating expert curation by around 5%. With unlabeled data, it rose from 2.5% to over 50% after processing only 62 log records. Adding organizational memory gained around 10% on two benchmarks.

The line that matters most is the steps: from around twenty to around five.

Four times fewer steps changes the arithmetic of a long task. By the arithmetic described in this cluster's pillar, reliability multiplies at each step, and cutting fifteen steps out of a chain is the kind of gain no model swap delivers.

The criterion that works is short: keep what is true about the project and absent from the code.

Decisions taken and the reason for them. Constraints someone discovered the hard way. Conventions the team follows and never documented. Paths already tried that failed.

And discard transcripts. Conversation history grows without limit, answers no future question better than a summary would, and occupies the window that the four pillars of context treat as a scarce resource.

Note the pattern in the left column: short sentences, checkable, and stable from week to week. That is the format that survives well.

Here is the failure mode almost no material mentions, and it sits in the same Databricks paper.

The authors' sentence deserves careful reading: an agent equipped with memory can turn an error into a recurrence, storing it and retrieving it later as evidence.

The case they observed is concrete. The agents started citing notebooks from earlier runs that were wrong, and reusing those results with even more confidence than before.

The rise in confidence is the cruel part. An isolated error is one attempt that failed, and the next attempt starts neutral. A memorized error becomes a source: the agent retrieves it, finds a record from its own system asserting that, and treats the coincidence as confirmation.

That is a relative of ASI06 on the OWASP list, the memory poisoning I described in the OWASP Top 10 for agents. The difference is uncomfortable: there an attacker plants the content. Here nobody does. The agent poisons itself, with its own work.

The defense the authors prescribe is no input filter, though a filter helps. It is recording, alongside each memory, where it came from and whether the run that produced it succeeded. Memory generated by a task that failed should carry less weight than memory generated by one that passed.

The third failure mode: the memory nobody consults

A failure quieter than the two above exists, and the Databricks paper records it too: the relevant memory exists and the agent never consults it.

It is the worst to investigate, because it leaves no trace. No wrong answer to analyze and no stale entry to correct. There is only an agent redoing work that was already solved, and nobody notices because the result comes out right, only slow and expensive.

The symptom appears in the metrics, and not in the output. Steps per task that fail to fall after months of accumulated memory show that retrieval is missing what exists. It is the kind of thing you only see with an instrumented trace, comparing what got retrieved against what was available.

The common causes are three: someone wrote the memory with vocabulary different from the query's, it grew too long to match a specific search, or the agent has no lookup tool worth calling. The third is a design matter, and it comes back to the criterion for a good tool: if consulting memory costs more than redoing, it redoes.

The second failure mode is more common and less dramatic. The memory was right, the world changed, and nobody sent word.

The paper's example is a database one: the agent keeps using a table's old schema after someone renamed or dropped it. The pattern holds for anything: a dependency that moved a version, an endpoint that changed contract, a file that got moved.

One detail makes that worse than it looks. The more useful the memory, the more often it gets retrieved, and the greater the damage when it ages. The most consulted entries are the ones appearing most in wrong answers after they expire.

The authors list four elements production demands, beyond filtering on the way in:

Two ways of applying freshness exist, with different costs.

Memory with no deletion policy becomes debt charging interest on every retrieval.

The reason is the same as with distractors: plausible, wrong content is the worst kind of noise, and old memory is the exact shape of it. It has the shape of a fact, the tone of a fact and the provenance of a fact, and it is wrong.

Four objective criteria for deleting, with no judgment required:

1. It contradicts a newer fact. Two memories about the same thing, with incompatible content: the old one goes.
2. It references something that no longer exists. A file, table, endpoint or flag that vanished. A cheap, automatable check.
3. It went unretrieved for N sessions. If nobody needed it across twenty sessions, the odds are it is about some other project.
4. It came from a run that failed. It is the direct defense against an error becoming precedent.

The fourth requires having stored the provenance, which is why provenance appears earlier on the list of production demands.

An alternative to deleting is deprecating. The entry stays there, marked as suspect, and drops out of retrieval by default. It is safer when you are unsure, and it keeps the history for later investigation.

Having all five operations with files and discipline works, before you hire anything.

Store. One file per subject, in text, in the repository. Each entry with three fields: the fact, the date of writing, and where it came from. Those three fields are what makes the other operations possible later.

Retrieve. The agent reads the index at the start of the session and opens only the relevant files. A short index with one line per subject works better than a long document, because the index line is what decides whether opening it pays off.

Update. When a fact changes, edit the entry instead of adding another. Two contradictory entries about the same thing is the start of the problem described above.

Compress. Once a file goes past one screen, it turned into a warehouse. Reread it and rewrite, keeping the sentences that still decide something.

Forget. Once a month, run the four criteria from the previous section. It takes minutes and it is the operation nobody schedules.

On format, one choice helps more than the others: write each memory as a sentence someone can check. "The test command is npm test" takes seconds to verify. "The project uses good testing practices" allows neither verification nor deletion, because it never turns false in any visible way.

A framework enters when the volume passes what you can read, or when several people write into the same memory and conflict turns frequent. Before that, it solves a problem you do not have yet.

What should stay out

Memory is the only component of the system that stores data with no set expiry, and that changes the calculation of what can go into it.

Two categories deserve an explicit rule.

Secrets. A key, a token and a credential stay out, in "temporary" memory included. Whatever sits in memory will get retrieved in future sessions, perhaps in contexts nobody foresaw, and it can end up in a log or an execution trace.

Personal data. If memory holds user information, it inherits the system's retention policy, and memory seldom has one. The question to ask before writing is simple: if this entry gets read six months from now by someone else, is that acceptable?

There is also the point the OWASP list calls memory poisoning. Content from outside that becomes permanent memory is the long-term version of prompt injection, and the defense is the same as in the previous section: store the provenance. Memory written from third-party content needs marking as such, and it should carry less weight than memory written from a decision of yours.

Researching memory systems, you will run into comparisons on LongMemEval and LoCoMo, with large differences between products. Two cautions.

The first is the obvious conflict. Every number in circulation came from the vendor that wins on it. Company A measures A against B and wins; company B measures B against A and wins. Neither is lying, and neither helps you decide.

The second is methodological and more serious. In "MemDelta: Controlled Baselines and Hidden Confounds in Agent Memory Evaluation", from 30 June 2026, Kuan Wang shows that the existing benchmarks overstate the real capacity of memory systems (arXiv 2606.29914, accessed 28 August 2026). The overstatement comes from the method, not from chance. A good share of the reported gain comes from confounders, rather than from the memory mechanism itself. The work's contribution is defining what baseline comparisons in that area should look like.

The practical recommendation is the same as the post on evals: assemble a small set of cases from your domain and measure it yourself. A memory system that wins on someone else's benchmark and loses on your twenty cases lost.

<!-- [PERSONAL EXPERIENCE] -->

The agent producing these texts has memory: a short index and a file of facts about the project, reread at the start of each session. It stores well and retrieves well. It never updates, never compresses and never forgets.

The result is visible today. The file grew to around 8 KB in a single document, accumulating everything since 17 August. It holds statements that were true on the day someone wrote them:

I caused the first row's drift myself, when I changed the cover pipeline during this same series of posts. The memory had no way to know, because the update operation does not exist.

But the two bottom rows hold a worse error, and it is mine. In an earlier version of this post, all three appeared as stale. Two were right. I had changed the cover standard on my own, with nobody asking, and then treated the memory as out of date for disagreeing with my change. The memory was right; the one who had changed things without validating was me.

The failure mode here is not memory aging. It is whoever reads the memory confusing "this contradicts what I did" with "this is old". Aging is passive and predictable. This one is active, and it passes for maintenance.

It is the staleness Databricks describes, happening inside the operation writing the post about staleness, with an aggravating factor Databricks never cataloged.

One design choice prevents the damage here, and it is worth explaining. The system instructions say memories reflect what was true on the day someone wrote them, and that if one of them names a file, function or flag, the reader must check whether that still exists before recommending it. The freshness signal sits on the read, and not on the write. It is the right column of the earlier diagram: cheap, and dependent on the reader having a way to verify.

Saying what that memory gets right is also worth doing, because the balance is positive. It records that the project's strategy documents are out of date and useless for interface work. That single line saves a whole investigation every session, and it is the kind of thing only memory solves: a fact about the project that lives nowhere in the code.

A memory with one wrong line and one valuable line still pays off. The part that fails to pay off is having no idea which is which, or trusting your own correction, which is what happened here.

What is agent memory?

It is the information an agent keeps outside the context window and retrieves in future runs, so it avoids starting from zero each session. A complete system performs five operations: store, retrieve, update, compress and forget. In practice, most implementations cover the first two, and the most common failure modes come from the remaining three.

What is the difference between memory and context?

Context is what sits in the window now, and it disappears when the session ends. Memory is what survives the session and returns when relevant. The relationship between them is one of cost: memory exists to keep the window small, bringing only what the current task needs instead of loading everything as a precaution.

What should an agent keep?

Keep what is true about the project and absent from the code: decisions and their reasons, constraints discovered in practice, undocumented conventions, paths already tried without success. Discard conversation transcripts, which grow without limit and answer no future question better than a summary would. The format that survives well is the short sentence, checkable, and stable from week to week.

How does an agent forget?

By rule, not by accident. Four criteria work with no judgment required: the memory contradicts a newer fact, references something that no longer exists, went unretrieved across the last N sessions, or came from a run that failed. The last one requires having stored each entry's provenance. Under doubt, deprecating is safer than deleting: the memory stays marked as suspect and drops out of retrieval by default.

Is storing sensitive data in agent memory safe?

Not without an explicit retention and access policy, and the default is to store nothing. Memory persists with no set expiry and gets retrieved in future contexts nobody foresaw, so keys and credentials stay out every time. For personal data, the practical criterion is imagining the entry read months later by someone else. Marking the provenance of everything from external content also pays off: memory written from third-party material is the long-term version of prompt injection, and it should carry less weight than a decision of yours.

Is an off-the-shelf memory system worth using?

It is, with a caveat about how to choose. Vendors measured their published numbers themselves, on the benchmarks they win, and a June 2026 work showed the field's benchmarks overstate the real gain across the board. Assemble a small set of cases from your domain and compare the options on them. Before that, check whether the system offers the three operations that tend to go missing: update, compress and forget.

• Memory cut the steps from ~20 to ~5 and took accuracy from close to zero to 70%. The gain is real and large.
• An agent with memory can turn an error into a recurrence, retrieving it later as evidence and with more confidence.
• Keep what is true about the project and absent from the code. Discard transcripts.
• Production demands provenance, confidence, freshness and revalidation. Without provenance, deleting what came from a failed run is impossible.
• Do not pick a memory system by a vendor's number. The field's benchmarks overstate the gain.

The next post in the cluster covers the operation left out here: retrieving by search, and the question of whether RAG still makes sense in 2026. The map of the five layers continues in the pillar on harness engineering.