Why your agent needs an ontology, not a bigger context window

Every time context windows get longer, someone declares the memory problem solved. It is not, and the reason is structural rather than technical: a longer window lets an agent read more, and reading more is not the same as knowing anything. A pile of passages has no shape to check a claim against, no way to trace where a belief came from, and no mechanism for noticing that two of its own facts disagree.

What retrieval actually gives you

Retrieval-augmented generation is a good technique and it solves a real problem: getting relevant text in front of the model at the right moment. For question answering over documents it is often enough.

For an agent that acts, it leaves three gaps.

No identity. “Acme Sp. z o.o.” in one document and “ACME” in another are two strings. Nothing in a vector index knows they are the same customer, so the agent reasons about half a relationship at a time.

No provenance you can act on. The model can quote a passage, but you cannot delete one wrong fact. You can delete a document, and hope the belief it created goes with it.

No contradiction. Two passages disagreeing about a delivery date produce a fluent average. Nothing in the architecture treats disagreement as information.

What an ontology adds

An ontology is not a philosophy exercise. It is a declaration of the kinds of things in your world and how they may relate: a client, a commitment, a document, a deadline, and the fact that a commitment has exactly one owner and at least one deadline.

Three things follow immediately.

Questions become answerable. “Which commitments to this client are at risk?” is a traversal, not a search. The agent walks relations rather than hoping the right paragraph was retrieved.

Facts get identity. The client is one node, however many ways their name has been written. Everything attaches to that node, which is what makes the knowledge accumulate instead of pile up.

Wrongness becomes local. A bad fact is one edge. Delete it and the rest stands. Compare that to correcting a belief that lives, implicitly, inside a summary written six weeks ago.

The graph is where the agent’s memory actually lives

The ontology says what kinds of things exist. The graph holds the instances — this client, that commitment, this document, with edges between them and, crucially, provenance on every edge: what said so, when, and how confident we are.

Provenance is the part most designs skip and the part that matters most for something acting on your behalf. It is what lets an agent answer “why do you believe that”, and what lets you correct it precisely rather than by retraining your assistant through conversation.

The honest objection

Ontologies have a bad reputation, earned in the years when building one meant a committee, a year and a document nobody read. That failure was real.

What changed is who does the work. A model can propose an ontology from your actual documents, suggest extensions when it encounters something that does not fit, and flag contradictions for a human to resolve. The schema stops being a two-year project and becomes something that grows with use — while staying declared, versioned and inspectable, which is exactly what the old approach got right.

What this looks like in practice

The agent still uses a language model for everything models are good at: reading messy text, extracting structure, drafting, summarising, judging tone. What changes is where the results go.

Instead of a summary that lives in a chat log, the extraction becomes nodes and edges with provenance. Instead of the next question re-reading everything, it queries. Instead of the agent’s knowledge being whatever fits in the window today, it is a graph you can inspect, correct, back up and carry to whatever model exists in three years.

Context windows will keep growing, and that is genuinely useful. It just does not remove the need for the agent to know what it knows.