Benchmarks
Measured, not promised.
How we test Geniffy's memory: public datasets, judged the way the field judges them, with the method beside every figure.
The figures
Four numbers we watch.
Answer accuracyLoCoMo, MSC and support conversations, judged with the LongMemEval prompt
0.0%1
Time-aware questionsLoCoMo temporal questions: what was true, and when
0.0%1
False premises refusedQuestions about things nobody said, answered with "nothing supports that"
0.0%1
Quotes that match their sourceEvery memory keeps the words it came from, checked word for word
0.0%1
1. Measured on Geniffy's memory engine test harness in September 2026: LoCoMo, MSC and ABCD conversations and synthetic cases, 190 to 1,373 questions per figure, judged with the LongMemEval judge prompt.
Method
How they are measured.
- Datasets
- LoCoMo (very long conversations between two people), MSC (multi-session chat), ABCD (customer support dialogues), and synthetic cases for forgetting, secrets and instructions hidden in what is saved.
- Judging
- Each answer is judged with the LongMemEval judge prompt, blind to which system wrote it. A second model family judged a sample, to check the first.
- What counts
- Answer accuracy; questions about what was true when; questions about things nobody said, where the right answer is that nothing supports one; and whether each memory quotes its source word for word.
- Size
- Between 190 and 1,373 questions per figure.
Plainly
What these figures are not.
Not a promise for every question. They describe these datasets, on our test harness, at the date given; your data will differ.
Not a comparison. Figures that other teams publish use their own setups and judges, so we do not set ours beside them.
Not frozen. We measure again as the memory engine changes, and a figure here changes only with a new run behind it.