500 questions · accuracy
LongMemEval-S
Ahead of Mem0’s open-source release, run in the same setup. Mem0’s hosted platform reports 93.4 on its blog (opens in a new tab), 1.8 points above Alexein’s 91.6, and other vendors self-report higher still.
Benchmarks
Seven public benchmarks, set beside the best published results: where Alexein leads, where it’s level, and where it isn’t first yet.
Same answer model
1,540 questions
Accuracy in percent, highest first:
500 questions
Accuracy in percent, highest first:
Other vendors self-report higher still.
vs paper’s Table 1
401 questions
Accuracy in percent, highest first:
3.2× the paper’s best memory
1,000 questions
F1, highest first:
4.6 F1 behind HippoRAG 2.
Alexein’s results are self-reported. Published results as their sources report them.
+58.0
points, same model, better memory
GPT-4.1-mini when facts keep changing: 36.0 to 94.0
3.2×
the LoCoMo-Plus paper’s best memory
55.4 against A-Mem’s 17.20
#1
memory framework on Microsoft’s RHELM
ahead of memU, Mem0 and MemGPT; top 4 of 18 overall
70.57%
pass rate on BEAM’s million-token conversations
level with Mem0’s 70.14%, in Mem0’s own harness
01Five pairs· GPT-4.1-mini, both ways
GPT-4.1-mini answers the same questions twice: once reading the whole history, once reading the brief Alexein hands it. Only the memory changes, and in every pair the brief wins, from a fraction of the reading.
Scores out of 100.
MemoryAgentBench
Without Alexein · reads 262K tokens
With Alexein
Fact consolidation: 262K tokens of facts, overwritten as they go; answer with the latest. Accuracy (SubEM), no judge. MemoryAgentBench, arXiv 2507.05257 (opens in a new tab)
RHELM
Without Alexein · reads the whole history, in a 1M-token window
With Alexein
All 1,305 questions, with emails and documents in memory. RHELM’s own reader and judge prompts, GPT-4.1-mini judging. RHELM leaderboard, github.com/microsoft/RHELM (opens in a new tab)
LoCoMo
Without Alexein · reads 26,031 tokens
With Alexein
All 1,540 questions over multi-session chats. LLM judge. Mnemis, arXiv 2602.15313 (opens in a new tab)
MemoryAgentBench
Without Alexein · reads the whole context
With Alexein
RULER-QA1, accurate retrieval. Accuracy (SubEM), no judge. MemoryAgentBench, arXiv 2507.05257 (opens in a new tab)
MemoryAgentBench
Without Alexein · reads the whole book
With Alexein
DetectiveQA, all 10 books (71 questions). Accuracy. MemoryAgentBench, arXiv 2507.05257 (opens in a new tab)
Alexein’s results are self-reported; the whole-history scores are as their sources publish them.
02LoCoMo· 1,540 questions
Long conversations over many sessions, then questions about what was said. Every system here answers with GPT-4.1-mini, so the memory is what’s compared. Alexein’s 94.6 is ahead of MindMemOS, Mnemis and EverOS, but only 0.57 points above the first of them; judges vary by source and the answer key has known errors, so read the top of this board as close.
LLM-judge accuracy. Alexein’s is self-reported; the rest as MindMemOS, arXiv 2608.12428 (opens in a new tab) and Mnemis, arXiv 2602.15313 (opens in a new tab) publish them. Judges follow each source; Alexein’s answers are judged by GPT-4o-mini, majority of three.
LoCoMo’s answer key has known errors, listed in an audit of it (opens in a new tab); they affect every top system’s score, not only Alexein’s.
03LoCoMo-Plus· 401 questions
Its questions hinge on something said long before, and never said in so many words. The paper’s memory systems answer with GPT-4o; the models with no memory read the whole conversation.
3.2×
55.4 against A-Mem’s 17.20, the best memory system in the LoCoMo-Plus paper.
Binary-judge accuracy. Alexein’s is self-reported; the rest are the LoCoMo-Plus paper’s Table 1 (opens in a new tab), and the comparison is to that table. A later paper reports a higher score from its own runs.
04RHELM· Microsoft · 1,305 questions
A year in ten people’s lives: their chats with an assistant, their emails and documents, and 1,305 questions about all of it. With the same reader as the paper’s memory frameworks, GPT-4.1-mini, Alexein ranks above every one of them, on both of the leaderboard’s tables.
The highest-ranked memory framework on both tables, ahead of memU, Mem0 and MemGPT, the paper’s memory frameworks, each reading with GPT-4.1-mini.
Above it: frontier models reading the whole history in a 1M-token window, and on the chats-only table one retrieval setup, 36.2 against 35.6, within the ±2.6 margin.
Averages as published on the RHELM leaderboard (opens in a new tab) (paper: arXiv 2605.31086 (opens in a new tab)); Alexein’s is self-reported. The margin on an average is ±2.6 at 95%.
05MemoryAgentBench· Fact consolidation
A long run of facts, each one overwritten as it goes, then questions that ask for the latest. Reading all of it, GPT-4.1-mini gets 36.0. With Alexein, the same model gets 94.0, ahead of MindMemOS (+ Dreaming) at 92.0 with gpt-5-mini.
Accuracy on single-hop fact consolidation; each row answers with the model beside it. Alexein’s is self-reported, from 100 questions per fact-consolidation set; published rows from MindMemOS, arXiv 2608.12428 (opens in a new tab) and MemoryAgentBench, arXiv 2507.05257 (opens in a new tab).
06BEAM· 1M-token split · 700 questions
35 conversations of about a million tokens each, every one larger than GPT-5’s 400,000-token window. Alexein hands the answer model a brief instead, and answers level with Mem0.
The same 700 questions, GPT-5 answering and judging with Mem0’s published harness (opens in a new tab), no hints.
| Alexein | Mem0 | |
|---|---|---|
| Mean score | 63.49 | 64.09 |
| Pass rate | 70.57% | 70.14% |
Mem0 is ahead on the mean by 0.60, Alexein on the pass rate by 0.43 points: less than a point apart, either way.
Alexein’s figures are self-reported; Mem0’s are from its published harness (opens in a new tab). The benchmark: BEAM, arXiv 2510.27246 (opens in a new tab).
Hindsight (opens in a new tab) and Exabase (opens in a new tab) report BEAM accuracy from their own setups; the only like-for-like accuracy comparison here is the one with Mem0.
07Not first, yet· LongMemEval-S · MuSiQue
Two benchmarks where other systems’ published scores are ahead of Alexein’s, shown as plainly as the rest. On BEAM, the one like-for-like comparison is level with Mem0; on RHELM, frontier models reading the whole history score higher.
500 questions · accuracy
Ahead of Mem0’s open-source release, run in the same setup. Mem0’s hosted platform reports 93.4 on its blog (opens in a new tab), 1.8 points above Alexein’s 91.6, and other vendors self-report higher still.
1,000 questions · F1
F1 44.0 (EM 33.9), 4.6 behind HippoRAG 2. Published rows answer with Llama-3.3-70B (HippoRAG 2’s Table 2 (opens in a new tab)).
Alexein’s results are self-reported; published results as their sources report them.
08All results
Alexein’s figure on each benchmark beside the best comparable published ones, then RHELM’s two tables side by side.
| Benchmark | Questions | Alexein | Best comparable published results |
|---|---|---|---|
| LoCoMoAccuracy, LLM judge | 1,540 | 94.6 | MindMemOS 94.03Mnemis 93.9EverOS 93.05Every row answers with GPT-4.1-mini |
| LoCoMo-PlusAccuracy, binary judge | 401 | 55.4 | A-Mem 17.20Mem0 15.80SeCom 14.90The paper’s Table 1; its memory systems answer with GPT-4o |
| LongMemEval-SAccuracy | 500 | 91.6 | Mem0 Platform 93.4Mem0 OSS 91.0 (same setup)Mem0 Platform’s figure is from Mem0’s blog |
| MuSiQueF1 | 1,000 | 44.0EM 33.9 | HippoRAG 2 48.6Published rows answer with Llama-3.3-70B |
| MemoryAgentBench, fact consolidationAccuracy | 100 per set | 94.0 | MindMemOS (+ Dreaming) 92.0 (gpt-5-mini)GPT-4.1-mini, whole context 36.0Alexein answers with GPT-4.1-mini |
| BEAM, 1M tokensMean score, and pass rate | 700 | 63.4970.57% pass | Mem0 64.09 (70.14% pass)GPT-5 answering and judging, Mem0’s published harness |
LoCoMo1,540 questions
Accuracy, LLM judge
Every row answers with GPT-4.1-mini
LoCoMo-Plus401 questions
Accuracy, binary judge
The paper’s Table 1; its memory systems answer with GPT-4o
LongMemEval-S500 questions
Accuracy
Mem0 Platform’s figure is from Mem0’s blog
MuSiQue1,000 questions
F1
Published rows answer with Llama-3.3-70B
MemoryAgentBench, fact consolidation100 questions per set
Accuracy
Alexein answers with GPT-4.1-mini
BEAM, 1M tokens700 questions
Mean score, and pass rate
GPT-5 answering and judging, Mem0’s published harness
| Without external sourcesChats only | With external sourcesChats, emails and documents | |
|---|---|---|
| Alexein’s average | 35.6 | 48.5 |
| Rank, of 18 | 4th | 3rd |
| Best other memory framework | memU 23.1 | memU 33.6 |
| Alexein’s lead over it, same reader | +12.5 | +14.9 |
| Best overall | GPT-5.5 57.01M-token context | GPT-5.5 73.31M-token context |
Without external sourcesChats only
With external sourcesChats, emails and documents
09Methodology & sources
The memory behind these results, a few lines of code away.