Public benchmark · rerun September 28, 2026
Memory claims, measured.
On the benchmark-native LongMemEval retrieval task, hippocampOS finds labelled evidence in its first five results 97.2% of the time without an LLM, provider, or network call. Explicit relative dates and first-person user statements now inform ranking; evaluation labels remain excluded.
Headline
97.2%
LongMemEval scoped Recall@5
Providers
None
Versus Raw
+0.6 pp
12,865
public questions
3
benchmark families
10
retrieved results
0
provider or network calls
Latest results
Retrieval, before generation.
Recall@K asks whether any labelled evidence appears in the first K results. MRR rewards placing the first correct result nearer the top.
LoCoMo
global1,986 questions · 272 dialogue-session documents
R@1
69.7%
R@5
90.5%
R@10
94.7%
MRR
0.784
Median
<1 ms
P95
<1 ms
The exact-sequence reranker improves early precision while preserving the original top-10 candidate set.
LongMemEval-S
scoped500 questions · benchmark-native evidence-session scope
R@1
87.2%
R@5
97.2%
R@10
98.6%
MRR
0.913
Median
<1 ms
P95
<1 ms
Full 500-question scoped run with explicit relative-date grounding and first-person user-evidence weighting.
ConvoMem sample
global10,379 questions · 12,394 dialogue-evidence documents
R@1
43.2%
R@5
60.0%
R@10
65.6%
MRR
0.504
Median
21 ms
P95
47 ms
P95 latency is 47 ms. Assistant-fact evidence reaches 86.7% R@10; implicit connections remain weakest at 28.2%.
Current results
Latest measured values from the public benchmark run on September 28, 2026.
| Benchmark | R@1 | R@5 | R@10 | MRR | Median | P95 |
|---|---|---|---|---|---|---|
| LoCoMo | 69.69% | 90.48% | 94.71% | 0.7837 | <1 ms | <1 ms |
| LongMemEval-S (scoped) | 87.20% | 97.20% | 98.60% | 0.9134 | <1 ms | <1 ms |
| ConvoMem sample | 43.23% | 59.95% | 65.62% | 0.5036 | 21 ms | 47 ms |
Global-corpus LongMemEval is the stress test.
This separate 19,195-session run removes the question-supplied scope. Direct assistant facts and explicit updates retrieve well; preferences, cross-session synthesis, and temporal evidence remain weak under global interference. These category values do not power the 97.2% scoped headline.
Single-session assistantR@5 94.6% · R@10 96.4%
Knowledge updateR@5 75.6% · R@10 80.8%
Single-session userR@5 37.1% · R@10 45.7%
Temporal reasoningR@5 41.4% · R@10 49.6%
Multi-sessionR@5 36.8% · R@10 42.1%
Single-session preferenceR@5 6.7% · R@10 10.0%
R@5 R@10 remainder
A same-metric reference point.
Same LongMemEval-S dataset. Same 500 questions. Same evidence-session recall-any@5 metric. Both runs use each question’s supplied retrieval scope and require no LLM. Retrieval implementations differ. This run uses no evaluation category labels in ranking; question dates can inform time scoring.
hippocampOS
97.2%
R@5 · full 500 questions
measured difference +0.6 percentage points
hippocampOS depth
98.6%
R@10
0.9134
MRR
0
provider calls
Important boundary
MemPalace also reports 98.4% R@5 for its Hybrid v4 system on a held-out 450-question split. That is a stronger published number under a different experimental condition; it is not the Raw baseline compared above.
Harder stress test
48.8% R@5 across 19,195 unique sessions.
The global run removes each question’s supplied haystack and searches the entire parsed corpus. It tests interference at scale, not the benchmark-native condition used in the direct comparison.
Useful context, not a leaderboard.
hippocampOS and MemPalace Raw use the same full 500-question retrieval protocol. MemPalace Hybrid v4 uses a held-out split; the remaining systems publish generated-answer accuracy or use different datasets.
| System | Published result | Benchmark | Metric | Conditions |
|---|---|---|---|---|
| hippocampOS | 97.2% R@5 | LongMemEval-S | Evidence-session recall-any@5 | Full 500 questions · benchmark-native scope · no evaluation labels · no LLM |
| MemPalace Raw | 96.6% R@5 | LongMemEval-S | Evidence-session recall-any@5 | Full 500 questions · benchmark-native scope · no LLM |
| MemPalace | 98.4% R@5 | LongMemEval-S | Evidence-session recall@5 | Hybrid v4 · held-out 450-question split |
| Mem0 v3 | 94.8% overall | LongMemEval | Generated-answer evaluation at top 50 | Managed memory · LLM answerer and judge |
| Zep / Graphiti | 94.8% | Deep Memory Retrieval | Answer accuracy | Different dataset and evaluation pipeline |
Methodology
Reproducible by design.
- Public datasets are read from local files; no private user corpus is included.
- Hybrid mode combines deterministic lexical and structural retrieval signals.
- Benchmark-native scope preserves shared sessions in each supplied haystack; this is the protocol used for the 97.2% comparison.
- Global scope searches 19,195 deduplicated sessions without question-provided filtering or dates; its 48.8% R@5 is a separate stress test.
- Top K is 10. Seed 42 is recorded for reproduction; this runner does not shuffle questions.
- The offline harness makes no network, OpenAI, Anthropic, database, Redis, or storage calls.
- Per-question latency excludes dataset parsing and index construction; it is not live API latency.
- Gold evidence IDs are used only after retrieval to calculate metrics.
- Dataset-supplied question categories are used only in the result breakdown, never in ranking.
- Explicit relative dates use the supplied question date; first-person questions give bounded weight to matching user turns.
- Earlier 97.0% and interim 96.2% headlines are superseded by corrected session scope and scoring.
- Scores are retrieval recall, not generated-answer correctness, citation quality, or hallucination rate.
- Date and speaker weights were selected on a 403-question development split; the remaining 97 were checked once, but this public dataset was inspected in earlier work and is not an independent held-out test.