What We Learned Evaluating Agent Memory:The Results (Part 2)

A detailed experimental evaluation of Couchbase Agent Memory across three benchmarks (LME-S, LME-M, LoCoMo) reveals several non-obvious findings. Raw conversation messages outperform LLM-generated summaries overall, especially for questions requiring exact entity matching. Optimal retrieval depth (k) varies by haystack scale — k=10 for small, k=20 for large — and more context beyond the optimum degrades performance. Temporal reasoning is the weakest category but improves ~7x by simply prepending session dates to stored memories at ingestion time. BM25 reranking with large initial pools hurts performance, while hybrid search improves temporal recall at the cost of multi-session precision. The key takeaway: agent memory has a distinct design space from document RAG, no single configuration is universally optimal, and systems tuned on small benchmarks may be misconfigured at production scale.

Source: https://www.couchbase.com/blog/what-we-learned-evaluating-agent-memorythe-results-part-2. 8 Sync News only summarizes and links out; content copyright belongs to the authors and original sources.