Research

Temporal validity for agent memory

Two papers are on arXiv. The rest of the program maps scope, logic bulk, and world-knowledge transfer. Experiments run locally and deterministically. Published papers are research, not product guarantees.

Paper 1 · arXiv:2606.26511 Paper 2 · arXiv:2608.20685 Read the blog
0.59
Cosine AUROC
Contradiction vs duplicate — near chance
~0%
Stale-fact error
MemStrata when forced to answer
0.95–1.0
Evolving accuracy
Four marker-free evolving benchmarks
~2.1s
Retrieval latency
Embedding floor — no LLM on read path
Published

Two papers on arXiv

Paper 1 · Published

1 Published arXiv:2606.26511 arXiv preprint · cs.CL / cs.AI

Temporal Validity in Retrieval Memory: Eliminating Stale-Fact Errors for AI Agents over Evolving Knowledge

Deterministic supersession that RAG cannot match by construction

RAG gives agents access to accumulated knowledge but has no model of time. When a fact changes, cosine similarity surfaces both stale and current values nearly equally (AUROC 0.59 for contradiction vs duplicate). MemStrata stores facts like RAG, then retires contradicted values with a deterministic (subject, relation, object) supersession rule in a bi-temporal ledger — no similarity threshold, no LLM on the read path. Across six local benchmarks with a 7B model, MemStrata ties RAG on static knowledge and reaches 0.95–1.00 accuracy on evolving knowledge where RAG reaches 0.20–0.47. Stale-fact-error drops from 15–40% (RAG, when forced to answer) to ~0%.

  • Cosine AUROC 0.59 — similarity cannot separate contradictions from duplicates
  • Evolving knowledge accuracy 0.95–1.00 vs RAG 0.20–0.47
  • Stale-fact-error ~0% vs RAG 15–40% when required to answer
  • Retrieval ~2.1s (embedding floor) vs ~16–18s LLM-rerank baselines
arXiv abstract Download PDF Related blog

Paper 2 · Published

2 Published arXiv:2608.20685 arXiv preprint · cs.SE / cs.AI

Temporal Validity on Real Software Histories: Eliminating Stale-Fact Errors in Code-Assistant Memory over GitHub Fixes

130 marker-free atomic transitions from 707 real SWE-bench GitHub fixes

Paper 1 showed deterministic (subject, relation, object) supersession on synthetic single-value benchmarks. Paper 2 validates it on real software history. From 707 SWE-bench Lite and Verified GitHub issues, 130 clean atomic state transitions are extracted and rendered marker-free. MemStrata reaches 0.91 answer accuracy versus RAG at 0.57-0.59. When forced to answer, RAG serves the superseded value 36-38% of the time (an LLM reranker does not help) while MemStrata drives this to about 0, at RAG retrieval latency (about 2.1 s versus about 18 s for the reranker). Scope is explicit: only about 18% of real fixes are clean atomic transitions.

  • 130 marker-free atomic transitions extracted from 707 real GitHub fixes
  • Answer accuracy 0.91 versus RAG 0.57-0.59
  • Forced-answer stale-fact error about 0 versus RAG 36-38%
  • About 18% of real fixes are this clean atomic class; the rest is follow-on work
arXiv abstract Download PDF Related blog
In progress

Drafts and follow-on papers

Papers 3 to 5 are not on arXiv yet. They stay labelled as draft or in progress.

3 Draft

Where Temporal Memory Applies: A Taxonomy of Code Evolution

Measuring the value-change ceiling on 707 real GitHub fixes

Papers 1–2 show deterministic supersession eliminates stale-fact errors on clean value changes. Paper 3 maps scope on real code: of 707 SWE-bench fixes, ~19% are native value-changes handleable by the ledger; a contract extractor lifts the ceiling to only ~24%. The remaining ~76% is logic/control-flow that does not reduce to a triple. Recency-aware RAG — including “pick newest chunk” LLM rerankers — fails to match the ledger; a recency-adversarial re-read of old values exposes a sharp boundary.

  • ~19% native value-change ceiling over 707 real fixes
  • Contract extraction lifts to only ~24% overall
  • ~76% logic bulk is out of triple extraction scope
  • Recency baselines do not substitute for a temporal ledger
taxonomycode evolutionvalue-change ceilingSWE-bench
Related blog →
4 Draft

Structural-Staleness-Aware Retrieval for the Un-Extractable Majority

Detect-and-flag for the ~76% logic bulk — without overclaiming extraction

Paper 4 answers the reviewer question left by Paper 3: what about the 76%? The honest answer is not “extract logic too,” but detect structural change and flag staleness at retrieval. Scope-level CST checksums mark logic-stale scopes; retrieval warns the model that code was modified since indexing. An identity layer (entity UUIDs + FQN resolution) productionizes renames without mutating ledger history. All mechanisms stay off the hot path or O(1).

  • Scope-level structural checksums for logic-staleness detection
  • Retrieval-time warnings instead of fake triple extraction
  • Entity UUID identity layer for renames over time
  • Moat-safe: does not mutate the CERTAIN ledger spine
structural stalenessdetect-and-flagtree-sitteridentity
Related blog →
5 In progress

Generalized Temporal Context: Value Supersession Beyond Code

Does the mechanism transfer to world knowledge?

Paper 5 is the breadth thesis and is not yet on arXiv. On synthetic marker-free world-knowledge scenarios, temporal_v6 reaches 0.85 accuracy / 0.075 stale with LLM extraction, and 1.000 / 0.000 with oracle gold triples. Same-stack LongMemEval-500 with a local 7B reader is a weak-baseline floor (naive RAG 0.252), not comparable to frontier published scores. A sealed n=500 local run with qwen3.8:27b (think off) scored 0.626 versus 0.702 on the gpt-4o-mini R230 stack. That is a research measurement, not a product claim.

  • Oracle synth: 1.000 acc / 0.000 stale (mechanism holds off-code)
  • LLM-extract residual is extraction quality, not supersession
  • Same-stack LME-500 7B floor: naive RAG 0.252; published competitor scores use different harnesses
  • qwen3.8:27b think-off n=500: 0.626 vs gpt-4o-mini stack 0.702 (research run, not a product SLA)
world knowledgegeneralizationTEMPOas-of queries
Related blog →

How we evaluate

Experiments run locally and deterministically on consumer hardware: Qwen2.5-Coder-7B, nomic-embed-text, temperature 0, fixed seeds, no network. We report stale-fact-error separately from accuracy, enforce marker-free evolving benchmarks, and compare competitors on the same stack when claiming architectural wins.

Citations: Yadav, N. (2026). Temporal Validity in Retrieval Memory. arXiv:2606.26511. Yadav, N. (2026). Temporal Validity on Real Software Histories. arXiv:2608.20685.

From the blog

Explainers & deep dives

All posts →

2026-08-31 · 7 min

MemStrata Coder with VS Code, Cursor, Windsurf, and Terminal Agents

2026-08-21 · 8 min

Temporal Validity on Real GitHub Fixes: Paper 2 Is on arXiv

2026-08-15 · 9 min

Why LongMemEval Scores Are Not Comparable Across Memory Products