# Recall without summarizing: agent memory on LoCoMo and LongMemEval

An agent that works on a project for days has to find the right sentence at the right moment. We measured Logposa Graph against Entire Graph and Mem0 on 1,540 questions from the LoCoMo benchmark, then against Entire on 400 questions from LongMemEval. Same model to answer, same judges, same context budget.

• 90.9% correct answers on LoCoMo. Entire: 90.3%. Mem0 OSS: 89.2%. The gap with Entire is not significant.
• 21 ms per search, median. Entire: 1.17 s. Same machine, each query in its own process.
• 0 model calls to index. Mem0 OSS read 5.7 million tokens to index the 272 sessions.
• 100% of passages copied word for word, with their file and lines. The benchmark checks this on every LoCoMo question.

Two ways to remember

There are two main ways to give an agent a memory.

• Summarize on the way in. A language model reads each session and draws short memories from it. This is Mem0’s approach. Every write costs a model call, and whatever the summary leaves out is gone.
• Keep the text and search when the question comes. Every exchange is kept as it was written. This is the approach of Logposa and Entire. Nothing is lost, but you have to choose well what you show the model that answers.

Logposa takes the second path, without a language model: a classic text index (BM25) that is deterministic. Same input, same result, every time.

The rules

• Same rules for everyone. The same model reads the context and answers (GPT-6-Luna, medium effort). The same judges grade.
• Same budget. At most 64 KiB of sources per question on LoCoMo and 32 KiB on LongMemEval, for every engine.
• Tuning and reserve kept apart. On LoCoMo, 233 questions are used for tuning. The other 1,307 are reserved: never read to tune the method.
• Two judges. The J-score accepts an answer that is partial but correct. The strict judge requires every expected element.
• Everything is archived: prompts, exact passages, answers, judgments, and fingerprints of the files and binaries.

Accuracy: neck and neck with Entire

On LoCoMo, Logposa and Entire answer correctly almost equally often. The measured gap leans toward Logposa, but it is too small to conclude. Under the strict judge, Logposa is clearly ahead of Mem0 OSS.

LoCoMo, 64 KiB	Logposa	Entire	Mem0 OSS
J-score, 1,540 questions	90.91%	90.26%	89.16%
J-score, 1,307 reserved	90.51%	89.98%	89.06%
Strict, 1,540 questions	74.48%	74.29%	70.65%
Strict, 1,307 reserved	74.29%	73.60%	71.00%
Evidence present in the context	94.90%	93.45%	not measurable

Logposa against Entire: the gap is not significant (McNemar, p = 0.40 on the J-score). Logposa against Mem0 OSS, strict judge on the reserve: p = 0.006. On the J-score, the gap with Mem0 is small and fragile (p = 0.099 on the reserve), so we do not claim it. Tests are per question and not corrected for multiple comparisons.

For each question, LoCoMo names the passage that holds the answer. Logposa’s context contains that passage a little more often than Entire’s. For Mem0 the measure is impossible: its memories are rewritten and do not keep the ID of the passage they came from.

Speed: tens of times faster than Entire

This is where the gap is clear. Each row has its own measurement setup, so we do not mix them.

Measure	Setup	Logposa	Entire
LoCoMo · search, median	same machine, one process per query	21 ms	1,173 ms
LoCoMo · search, index already loaded	466 paired queries	1.7 ms	742 ms
LongMemEval · index one history, median	four processes in parallel	13.8 ms	1,307 ms
LongMemEval · search, median	four processes in parallel, command-line startup included	60 ms	2,803 ms

For an agent, this changes how memory gets used. At 21 ms, it can search at every step of a task. At more than a second, every search is felt.

Indexing without a language model

Logposa builds its index without calling a model. Mem0, in its open source version, calls one for every session. On LoCoMo’s 272 sessions, its extraction had the model read 5,738,919 tokens and write 176,646. That is about 21,100 tokens read per session.

That cost is paid on every new conversation, whether the memory is used or not. For Logposa it is zero, and nothing goes to a third party at indexing time.

Back to the source

When an agent answers, you need to be able to ask: where does this answer come from? Logposa returns each passage with its file and its lines. The model that answers gets the exact text of the conversation, not a rewording:

(Monday, May 08, 2023) [session_1.md:1-17]
# Conversation session_1 | 1:56 pm on 8 May, 2023

## user: Caroline: Hey Mel! Good to see you! How have you been? [D1:1; 1:56 pm on 8 May, 2023]

Each block carries its file and its lines. Each line carries the ID of its turn and its date. Our benchmark checks that every passage is an exact copy of the source: a passage that is not stops the run.

• An answer can be checked down to the line. You reread the original passage, exactly as it was written.
• Nothing goes to a third party to index. The index is built without a language model. Only the model that answers sees the chosen passages, at question time.
• A summarized memory loses its source. In our test, the memories Mem0 rewrote did not keep the ID of the original passage. You can no longer show the exact sentence the answer rests on.

For a team under SOC 2, or in finance or health, this is useful audit evidence.

Processing integrity and confidentiality can be shown line by line. Logposa does not make anyone compliant on its own. It gives the auditor evidence they can verify.

What Mem0 does well

Mem0 reads less text: 51,378 bytes per question on average, against 65,457 for Logposa and 55,874 for Entire. On LoCoMo, its memories are so short that its whole store fits in the context for 88% of questions. It pays for that advantage at write time.

Mem0 also resolves relative dates when it extracts. In one case we reviewed, “yesterday”, written on January 29, had become the 28th. That is a good idea. We want to do the same without a language model, by marking the computed date as derived and keeping its source line.

We tested Mem0 open source, with the same model to extract, read and judge. This is not the Mem0 platform: its published numbers do not compare with ours.

A second benchmark: LongMemEval

One benchmark can mislead. We replayed the comparison with Entire on LongMemEval, in its cleaned version from September 2025 (LongMemEval_S). Each question there comes with its own long history.

LongMemEval, 400 reserved, 32 KiB	Logposa	Entire
J-score	72.00%	70.75%
Strict	64.75%	62.25%

The gap is not significant (sign test: p = 0.63 on the J-score, 0.28 under the strict judge). The J-score gap comes from the 25 questions where the right answer is “I don’t know”. This is not the official LongMemEval score: our judges are LoCoMo’s. Both engines send almost the same amount of text.

Same conclusion as on LoCoMo: accuracy neck and neck, search much faster.

What we don’t claim

• Being more accurate than Entire. The measured gap is too small to conclude, on both benchmarks.
• Reading less text. Logposa fills its whole budget, more than Entire and Mem0. A leaner assembler failed our tests: it lost more evidence than it gained. We dropped it.
• Being faster than Mem0. Our measurement setups differ too much to compare.

What we learned

Finding the evidence is not enough. Logposa puts it in the context more often, but does not answer correctly clearly more often. The bottleneck is reading, not search. That is where we are working now: presenting what was found better, without losing the source line.

Method

LoCoMo: the official file, checked by its SHA-256 fingerprint; 10 conversations, 1,540 questions. LongMemEval_S: 500 questions, 100 for tuning and 400 reserved, with the split frozen by fingerprint before any reading. Reader and judges: GPT-6-Luna, medium effort, identical for every engine. Mem0 open source, pinned SDK, extraction by the same model, local BGE embeddings.