Why RAG Cannot Tell Time: The Stale-Fact Error Problem for AI Agents

Retrieval-augmented generation (RAG) is the default memory layer for AI coding tools and long-running agents. It works brilliantly for recall: embed history, retrieve top-k, stay inside the context window. But the moment knowledge evolves — a function is renamed, a port changes, an API is restructured — RAG has a structural blind spot.

Both the old and new facts stay in the store. Their embeddings are nearly identical. A query about “the service port” surfaces 8000 and 8080 with similar cosine scores. The model has no principled way to know which is current, so it either abstains or — worse — answers with the superseded value at full confidence.

We call this the stale-fact error: when an agent is required to answer and serves a value that has already been replaced. On our evolving benchmarks, RAG commits this error 15–40% of the time when abstention is disabled. That is not a prompt bug. It is a missing model of time.

A natural instinct is to detect contradictions with cosine similarity and update the store. Empirically, that fails. On a calibrated set of 98 labeled pairs, cosine similarity separates contradictions from duplicates with AUROC 0.59 — near chance. Contradictions are on average more embedding-similar to the original than genuine rephrasings, because a value flip is a smaller edit than a paraphrase.

Maximum precision at any threshold sits around 0.67. The safety floor a automatic-update rule would need (~0.95) is unreachable. No similarity threshold can tell “this restates a fact” from “this supersedes a fact.”

Staleness detection must be structural: if two assertions share a (subject, relation) key but assert different objects, the newer one supersedes the older — independent of embedding distance. MemStrata implements that rule in a bi-temporal ledger: facts are retired, not deleted, with valid_from, valid_to, and superseded_by.

The result on that protocol: static recall stays competitive with the evaluated retrieval baseline, evolving-knowledge accuracy jumps to 0.95-1.00 where that baseline sits at 0.20-0.47, and stale-fact error collapses to about 0%. That failure class is what the measured baseline does not avoid by construction. Other retrieval designs can add time metadata; they were not the compared system.

Read the full technical treatment in our arXiv paper: arXiv:2606.26511.