Why RAG Cannot Tell Time: The Stale-Fact Error Problem for AI Agents
When knowledge evolves, RAG retrieves both stale and current facts with near-identical embeddings. Learn why stale-fact errors are structural — not a tuning problem — and what temporal validity changes.
The hidden failure mode of retrieval-augmented agents
Retrieval-augmented generation (RAG) is the default memory layer for AI coding tools and long-running agents. It works brilliantly for recall: embed history, retrieve top-k, stay inside the context window. But the moment knowledge evolves — a function is renamed, a port changes, an API is restructured — RAG has a structural blind spot.
Both the old and new facts stay in the store. Their embeddings are nearly identical. A query about “the service port” surfaces 8000 and 8080 with similar cosine scores. The model has no principled way to know which is current, so it either abstains or — worse — answers with the superseded value at full confidence.
We call this the stale-fact error: when an agent is required to answer and serves a value that has already been replaced. On our evolving benchmarks, RAG commits this error 15–40% of the time when abstention is disabled. That is not a prompt bug. It is a missing model of time.
Why “better similarity” cannot fix it
A natural instinct is to detect contradictions with cosine similarity and update the store. Empirically, that fails. On a calibrated set of 98 labeled pairs, cosine similarity separates contradictions from duplicates with AUROC 0.59 — near chance. Contradictions are on average more embedding-similar to the original than genuine rephrasings, because a value flip is a smaller edit than a paraphrase.
Maximum precision at any threshold sits around 0.67. The safety floor a automatic-update rule would need (~0.95) is unreachable. No similarity threshold can tell “this restates a fact” from “this supersedes a fact.”
What temporal validity requires
Staleness detection must be structural: if two assertions share a (subject, relation) key but assert different objects, the newer one supersedes the older — independent of embedding distance. MemStrata implements that rule in a bi-temporal ledger: facts are retired, not deleted, with valid_from, valid_to, and superseded_by.
The result: static recall stays competitive with RAG, evolving-knowledge accuracy jumps to 0.95–1.00 where RAG sits at 0.20–0.47, and stale-fact error collapses to ~0% — a failure class RAG cannot avoid by construction.
Read the full technical treatment in our arXiv paper: arXiv:2606.26511.