Agent memory after RAG: search, revise, forget

A long-running AI agent can keep collecting notes forever and still get worse at knowing you. Old plans compete with current ones. A one-off request hardens into a “preference.” A correction is stored alongside the mistake, and retrieval surfaces both. More long-term memory has made the agent less useful.

I think the old recipe for retrieval-augmented generation (RAG) is becoming a thing of the past for long-running agents. Fetch a few similar chunks, put them in the prompt, and hope the answer is there: that is too narrow a model of memory for an agent that can search, inspect, and change course. The harder design problem is what the agent should carry forward, how that information should change, and when it should stop shaping a response.

Different memories deserve different lifetimes

Consider three things a person might tell an assistant:

• “I prefer short answers.”
• “The meeting is on Thursday.”
• “I'm exploring a move to Paris.”

The first could be a durable preference, though it might change. The second is tied to an event and should quickly lose relevance. The third is tentative. Treating all three as equally permanent facts creates a misleading portrait of the person.

The distinction matters even when retrieval works perfectly. A vector search can bring the old meeting back into a prompt. It cannot, by itself, decide whether the meeting still matters or whether “exploring” ever became “moving.” Agent memory needs meaning, time, and a way to revise its own state.

Let the agent search instead of preloading an answer

Traditional RAG often treats retrieval as a single step before generation: rank stored chunks against a query and hand the top results to the model. That can be useful for a direct question over a stable collection. It is a poor default for memory that changes over time, where the important evidence may be a later correction, a dated note, or a relationship between several records.

A tool-using agent can work differently. It can look at a file tree, grep for an exact name or phrase, narrow by date, open the likely note, and follow a reference to another file. If the first search is inconclusive, it can form a better query and try again. The agent decides what to read next instead of accepting one preselected bundle of context.

Imagine asking why an earlier plan was abandoned. A similarity search might return the original plan because it shares the most words with the question. An agent could also search for later decisions, compare timestamps, inspect the correction, and say when the record is incomplete. Grep is useful here because exact names and decisions often leave precise traces. Semantic search remains useful when the wording differs; it becomes one tool in the investigation rather than the whole memory architecture.

This approach has costs. Repeated searches take time and tokens, and a badly organized store can send an agent down the wrong path. File names, dates, lightweight indexes, and bounded search tools help it navigate. The goal is to bring the smallest relevant evidence into working context, then make a decision about whether that evidence is still true.

A lifecycle for long-term agent memory

I would start with four actions: retain, revise, fade, and delete.

Retain when information is likely to improve future interactions and the person reasonably expects it to be remembered. Store enough context to know what was said, when, and with what level of certainty. “I might move” should not silently become “I live in Paris.”

Revise when new information corrects or supersedes old information. A good system should not simply append “I no longer want short answers” next to “I prefer short answers” and hope retrieval picks the right one. It should preserve the relationship between the statements and use the current one.

Fade when information might still be useful but has become less likely to matter. This could mean lowering its retrieval priority, asking before relying on it, or setting an expiry for a temporary plan. Temporal decay alone is not enough: a long-held preference may remain relevant, while a recent passing thought may never have deserved storage.

Delete when the person asks for removal, the information is no longer appropriate to retain, or a retention rule requires it. Fading is a relevance decision; deletion is a stronger commitment. A system should not claim to have forgotten something if it can still retrieve it elsewhere.

These are product choices as much as retrieval choices. The person should be able to see and correct consequential memories, and the agent should be able to say when it is uncertain.

Evaluate agent memory over time

Memory demos tend to reward an agent for recalling a detail months later. I would test the opposite too. Can it stop using a preference after a correction? Can it distinguish a temporary plan from a settled fact? Does it ask when two memories conflict? Does deletion actually prevent the information from resurfacing?

A useful evaluation would replay a sequence of conversations with changed facts, not a single question over a static archive. Measure both missed useful memories and harmful stale ones. The latter can be harder to notice: the answer may sound personal and confident while resting on something the user no longer believes.

Older memories might lose weight over time. That is one possible mechanism, not a complete theory of human memory. Search alone will not solve it either: an agent can find an obsolete fact perfectly and still misuse it. The engineering goal is to make an agent more accurate and more respectful over a long relationship. Remembering well includes knowing when yesterday's context should no longer speak for someone today.