Comparing MemStrata, Mem0, Graphiti and Letta: what the evidence can tell you

Choosing memory for an AI application starts with the job you need it to do. An assistant recalling a preference, a coding tool finding a changed API, and a support application answering from current policies may need different evidence and different tests.

MemStrata, Mem0, Graphiti and Letta often appear in the same discussion. A useful comparison needs more than a list of headline accuracy scores. You need to know which model answered, which questions it saw, what evidence it received, and how the answers were graded.

This article explains how to make that comparison and summarizes the relevant MemStrata author evaluations. We do not publish a verified common-protocol Mem0 or Graphiti score here. Their names identify alternatives a reader may be considering; they do not imply those products were included in every table below. How to read a protocol lives in Why LongMemEval scores are not comparable. Counts, judge, reader and run identifiers are on the evaluation methods page.

Build a small evaluation from questions your application really needs to answer. Include a preference stated once, a fact corrected later, an answer spread across multiple conversations, and a question the sources cannot answer.

For a changed fact, keep both the original statement and its correction in the test material. Ask for the current answer and inspect the source used. Also ask a historical question: what was believed before the correction? A system can answer the current question correctly while discarding information needed for the historical one.

For each question, record the answer, the selected evidence, and whether the evidence actually supports it. That makes a failure useful: you can distinguish missing retrieval from a model misreading evidence it already had.

In our author-run LongMemEval-S evaluation using Qwen 3.8 27B and a common GPT-5.5 judging protocol, the following configurations produced these results:

CL1 scored higher in this evaluation. These are results for the named configurations, not a general ranking of every deployment of these products. CL1 was developed using these benchmark families, so the result is development-exposed. A fresh holdout and customer-task evaluation would test how well the improvement transfers.

A configured token limit is also different from the evidence actually sent. Letta's 100k configuration did not send 100,000 evidence tokens on every question; the recorded mean was about 23,878 tokens. Any cost or efficiency comparison needs the observed token usage, reader work and latency, not just the configured ceiling.

The historical MemStrata R454 baseline scored 420/500, or 84.0%, under GPT-5.5 grading. Its reader pipeline differs from CL1. The numerical difference is 8.6 percentage points, but that comparison does not isolate a single retrieval change as the cause. See the method record.

On the evaluated 1,540-question LoCoMo population, covering categories 1-4, the earlier Qwen comparison produced:

MemStrata scored higher overall on this population, while Letta led the multi-hop category. The Zep result concerns a frozen snapshot, not the current live service. The retained comparison also has incomplete original wire receipts; that limits what can be claimed about exact request parity.

The later CL1 24k configuration scored 1,205/1,540, or 78.25%, on the same evaluated population. It is a separate, development-exposed configuration. These measurements do not establish end-to-end write costs, a live-service ranking, or a guaranteed customer accuracy rate. See the method record.

Keep the answering model, question set, grading rubric and relevant generation settings fixed when testing a retrieval change. Record any differences you cannot hold fixed. Measure the actual evidence sent, how often the system abstains correctly, and whether it preserves answers that were already correct.

Then test the operational questions: how long does initial indexing take on your data, how quickly do updates become available, what does an answer cost, and can a user inspect the source? Test source deletion and access boundaries as well as successful answers.

A strong result on one dataset is a reason to investigate a system. Your own task-level evaluation is what tells you whether it fits your application.

Read the research methods and results, try the Conversational setup guide, or discuss an integration pilot.