Methods

How the tests were run

Elvena was compared with three widely used AI memory and retrieval frameworks: Haystack, LlamaIndex and LangGraph. Every system used the same AI model, , the same stored records or conversation history, and the same final question. Final questions were ordinary work requests with no instruction to look for corrections. Scoring checks the facts in each returned answer against a fixed answer key.

All people, patients, companies, missions and documents are fictional. Times are measured wall-clock time to a returned answer. Index or archive setup time is measured separately and not included.

The three tests

• Changed records (10 cases). After a correction was approved, a stored copy of that review note was changed. The original source was left in place.
• Long conversations (6 cases). 197 complete chat turns per project. The owner corrects a mistake at turn 20; 177 more turns follow, including drafts that repeat the old mistake.
• Look-alike identities (15 matched cases). Two people share a name, or a convincing filename points to the wrong person.

Every recorded case shown on this site

18 cases with their exact questions, outcome and answer time for each system. The look-alike identity test totals above include all 15 matched cases; two of them are listed here.

What is not published

How Elvena works internally is not part of this site. Test records, full answer texts and source hashes are retained in the benchmark archive and are available for review on request.