Memory that survives the session

An assistant that starts every conversation from zero is a very good stranger. It is articulate, it is helpful, and it will ask you again next week which of your clients pays late. Memory is what separates a tool you operate from an agent you delegate to — and getting it right is mostly about deciding what not to keep.

Three kinds of memory, and only one of them is hard

Working memory is the current task: what we are doing right now, what has been tried, what the model needs in front of it. It lives in the context window, and it is supposed to be discarded. Nothing to solve here.

Semantic memory is what is true: this person is the finance contact, this supplier has a two-week lead time, this project ends in March. It belongs in the graph, with provenance, and it is the memory that makes an agent useful on day sixty.

Episodic memory is what happened: on Tuesday the agent drafted this, you rejected it, here is the reason you gave. This is the memory almost everyone skips, and it is the one that makes an agent improve rather than merely persist.

The hard part is not storing any of these. It is deciding what gets promoted from the first into the second and third — because an agent that writes everything down becomes unusable in about a month.

Promotion, not accumulation

The naive design appends everything: every message, every document, every conclusion. After a few weeks retrieval starts returning things that were true once, the agent’s confidence stops tracking reality, and the whole thing has to be wiped.

A better rule is that facts get promoted deliberately, and promotion is cheap to reverse.

Promote what is stable — a person’s role, a recurring constraint, a preference stated twice. Do not promote what is momentary — a mood, a one-off exception, an intermediate step. Promote with provenance: what said this, when, and how sure. And attach a review trigger to anything that looks like it might expire: a contract end date, a role that changes with reorganisations.

Forgetting is a feature

Human memory decays for good reasons: it keeps the useful and lets the rest fade, which is why we are not paralysed by the irrelevant.

An agent needs the same, deliberately implemented. Facts carry a confidence that decays without reinforcement. Contradicted facts are superseded rather than deleted, so the history of what the agent believed stays reconstructable. Anything not touched in a year gets flagged rather than silently trusted.

This is also where privacy stops being a policy document and becomes architecture. An agent that forgets by design holds less, so there is less to leak, less to subpoena and less to regret.

The things that should never be written down

Some information should pass through working memory and go no further. Health details mentioned in passing. What a colleague said about another colleague. Anything about a person who never agreed to be in your agent’s graph.

The right default is that promotion to long-term memory is an explicit decision with a rule behind it, not an automatic consequence of the agent having seen something. If a personal agent is going to be trusted with a life, the burden should be on writing things down rather than on remembering to delete them.

Why this is an argument for local

Everything above is easier to mean when the memory is on your own machine.

A graph you hold can be inspected, edited, backed up and destroyed. You can look at what your agent believes about your family and remove a node. That is not a feature a hosted assistant can credibly offer, no matter how good the privacy policy is — because the honest version of “delete my data” requires the data to have been somewhere you control in the first place.