RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915722378084352 |
|---|---|
| author | Bian, Haonan Yao, Zhiyuan Hu, Sen Xu, Zishan Zhang, Shaolei Guo, Yifu Yang, Ziliang Han, Xueran Wang, Huacan Chen, Ronghao |
| author_facet | Bian, Haonan Yao, Zhiyuan Hu, Sen Xu, Zishan Zhang, Shaolei Guo, Yifu Yang, Ziliang Han, Xueran Wang, Huacan Chen, Ronghao |
| contents | As Large Language Models (LLMs) evolve from static dialogue interfaces to autonomous general agents, effective memory is paramount to ensuring long-term consistency. However, existing benchmarks primarily focus on casual conversation or task-oriented dialogue, failing to capture **"long-term project-oriented"** interactions where agents must track evolving goals.
To bridge this gap, we introduce **RealMem**, the first benchmark grounded in realistic project scenarios. RealMem comprises over 2,000 cross-session dialogues across eleven scenarios, utilizing natural user queries for evaluation.
We propose a synthesis pipeline that integrates Project Foundation Construction, Multi-Agent Dialogue Generation, and Memory and Schedule Management to simulate the dynamic evolution of memory. Experiments reveal that current memory systems face significant challenges in managing the long-term project states and dynamic context dependencies inherent in real-world projects.
Our code and datasets are available at [https://github.com/AvatarMemory/RealMemBench](https://github.com/AvatarMemory/RealMemBench). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_06966 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction Bian, Haonan Yao, Zhiyuan Hu, Sen Xu, Zishan Zhang, Shaolei Guo, Yifu Yang, Ziliang Han, Xueran Wang, Huacan Chen, Ronghao Computation and Language Artificial Intelligence As Large Language Models (LLMs) evolve from static dialogue interfaces to autonomous general agents, effective memory is paramount to ensuring long-term consistency. However, existing benchmarks primarily focus on casual conversation or task-oriented dialogue, failing to capture **"long-term project-oriented"** interactions where agents must track evolving goals. To bridge this gap, we introduce **RealMem**, the first benchmark grounded in realistic project scenarios. RealMem comprises over 2,000 cross-session dialogues across eleven scenarios, utilizing natural user queries for evaluation. We propose a synthesis pipeline that integrates Project Foundation Construction, Multi-Agent Dialogue Generation, and Memory and Schedule Management to simulate the dynamic evolution of memory. Experiments reveal that current memory systems face significant challenges in managing the long-term project states and dynamic context dependencies inherent in real-world projects. Our code and datasets are available at [https://github.com/AvatarMemory/RealMemBench](https://github.com/AvatarMemory/RealMemBench). |
| title | RealMem: Benchmarking LLMs in Real-World Memory-Driven Interaction |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.06966 |