Public benchmark · rerun July 29, 2026
Memory claims, measured.
On the benchmark-native LongMemEval retrieval task, hippocampOS finds labelled evidence in its first five results 97.0% of the time—edging MemPalace Raw without an LLM, provider, or network call.
Headline
97.0%
LongMemEval scoped Recall@5
Providers
None
Raw edge
+0.4 pp
12,865
public questions
3
benchmark families
10
retrieved results
0
provider or network calls
Latest results
Retrieval, before generation.
Recall@K asks whether any labelled evidence appears in the first K results. MRR rewards placing the first correct result nearer the top.
LoCoMo
1,986 questions · 272 dialogue-session documents
R@1
69.7%
R@5
90.5%
R@10
94.7%
MRR
0.784
Median
<1 ms
P95
1 ms
The exact-sequence reranker improves early precision while preserving the original top-10 candidate set.
LongMemEval-S
500 questions · benchmark-native evidence-session scope
R@1
90.4%
R@5
97.0%
R@10
98.4%
MRR
0.934
Median
<1 ms
P95
2 ms
The same full 500-question scoped run used for the 97.0% headline and the direct MemPalace Raw comparison.
ConvoMem sample
10,379 questions · 12,394 dialogue-evidence documents
R@1
43.2%
R@5
60.0%
R@10
65.6%
MRR
0.504
Median
103 ms
P95
136 ms
P95 latency is 136 ms. Assistant-fact evidence reaches 86.7% R@10; implicit connections remain weakest at 28.2%.
Current results
Latest measured values from the public benchmark run on July 29, 2026.
| Benchmark | R@1 | R@5 | R@10 | MRR | Median | P95 |
|---|---|---|---|---|---|---|
| LoCoMo | 69.69% | 90.48% | 94.71% | 0.7837 | <1 ms | 1 ms |
| LongMemEval-S (scoped) | 90.40% | 97.00% | 98.40% | 0.9337 | <1 ms | 2 ms |
| ConvoMem sample | 43.23% | 59.95% | 65.62% | 0.5036 | 103 ms | 136 ms |
Global-corpus LongMemEval is the stress test.
This separate 19,195-session run removes the question-supplied scope. Direct assistant facts and explicit updates retrieve well; preferences, cross-session synthesis, and temporal evidence remain weak under global interference. These category values do not power the 97.0% scoped headline.
Single-session assistantR@5 94.6% · R@10 96.4%
Knowledge updateR@5 76.9% · R@10 82.0%
Single-session userR@5 55.7% · R@10 62.9%
Temporal reasoningR@5 46.6% · R@10 54.1%
Multi-sessionR@5 37.6% · R@10 46.6%
Single-session preferenceR@5 10.0% · R@10 16.7%
R@5 R@10 remainder
hippocampOS edges the raw baseline.
Same LongMemEval-S dataset. Same 500 questions. Same evidence-session recall-any@5 metric. Both runs use each question’s supplied retrieval scope and require no LLM. Under those aligned conditions, the comparison is meaningful.
higher
hippocampOS
97.0%
R@5 · full 500 questions
measured edge +0.4 percentage points
hippocampOS depth
98.4%
R@10
0.9337
MRR
0
provider calls
Important boundary
MemPalace also reports 98.4% R@5 for its Hybrid v4 system on a held-out 450-question split. That is a stronger published number under a different experimental condition; it is not the Raw baseline compared above.
Harder stress test
53.4% R@5 across all 19,195 sessions.
The global run removes each question’s supplied haystack and searches the entire parsed corpus. It tests interference at scale, not the benchmark-native condition used in the direct comparison.
Useful context, not a leaderboard.
hippocampOS and MemPalace Raw use the same full 500-question retrieval protocol. MemPalace Hybrid v4 uses a held-out split; the remaining systems publish generated-answer accuracy or use different datasets.
| System | Published result | Benchmark | Metric | Conditions |
|---|---|---|---|---|
| hippocampOS | 97.0% R@5 | LongMemEval-S | Evidence-session recall-any@5 | Full 500 questions · benchmark-native scope · no LLM |
| MemPalace Raw | 96.6% R@5 | LongMemEval-S | Evidence-session recall-any@5 | Full 500 questions · benchmark-native scope · no LLM |
| MemPalace | 98.4% R@5 | LongMemEval-S | Evidence-session recall@5 | Hybrid v4 · held-out 450-question split |
| Mem0 v3 | 94.8% overall | LongMemEval | Generated-answer evaluation at top 50 | Managed memory · LLM answerer and judge |
| Zep / Graphiti | 94.8% | Deep Memory Retrieval | Answer accuracy | Different dataset and evaluation pipeline |
Methodology
Reproducible by design.
- Public datasets are read from local files; no private user corpus is included.
- Hybrid mode combines deterministic lexical and structural retrieval signals.
- Benchmark-native scope searches each question’s supplied sessions; this is the protocol used for the 97.0% direct comparison.
- Global scope searches all 19,195 parsed sessions without question-provided filtering; its 53.4% R@5 is a separate stress test.
- Top K is 10 and the deterministic seed is 42.
- The offline harness makes no network, OpenAI, Anthropic, database, Redis, or storage calls.
- Scores are retrieval recall, not generated-answer correctness, citation quality, or hallucination rate.
- The July 29 global exact-sequence reranker was selected after inspecting these benchmark results; treat this run as evaluation-tuned, not held-out.