Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.10715 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908827659534336 |
|---|---|
| author | Li, Yifei Guo, Weidong Zhang, Lingling Xu, Rongman Huang, Muye Liu, Hui Xu, Lijiao Xu, Yu Liu, Jun |
| author_facet | Li, Yifei Guo, Weidong Zhang, Lingling Xu, Rongman Huang, Muye Liu, Hui Xu, Lijiao Xu, Yu Liu, Jun |
| contents | Long-term conversational memory is a core capability for LLM-based dialogue systems, yet existing benchmarks and evaluation protocols primarily focus on surface-level factual recall. In realistic interactions, appropriate responses often depend on implicit constraints such as user state, goals, or values that are not explicitly queried later. To evaluate this setting, we introduce \textbf{LoCoMo-Plus}, a benchmark for assessing cognitive memory under cue--trigger semantic disconnect, where models must retain and apply latent constraints across long conversational contexts. We further show that conventional string-matching metrics and explicit task-type prompting are misaligned with such scenarios, and propose a unified evaluation framework based on constraint consistency. Experiments across diverse backbone models, retrieval-based methods, and memory systems demonstrate that cognitive memory remains challenging and reveals failures not captured by existing benchmarks. Our code and evaluation framework are publicly available at: https://github.com/xjtuleeyf/Locomo-Plus. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_10715 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents Li, Yifei Guo, Weidong Zhang, Lingling Xu, Rongman Huang, Muye Liu, Hui Xu, Lijiao Xu, Yu Liu, Jun Computation and Language Artificial Intelligence Long-term conversational memory is a core capability for LLM-based dialogue systems, yet existing benchmarks and evaluation protocols primarily focus on surface-level factual recall. In realistic interactions, appropriate responses often depend on implicit constraints such as user state, goals, or values that are not explicitly queried later. To evaluate this setting, we introduce \textbf{LoCoMo-Plus}, a benchmark for assessing cognitive memory under cue--trigger semantic disconnect, where models must retain and apply latent constraints across long conversational contexts. We further show that conventional string-matching metrics and explicit task-type prompting are misaligned with such scenarios, and propose a unified evaluation framework based on constraint consistency. Experiments across diverse backbone models, retrieval-based methods, and memory systems demonstrate that cognitive memory remains challenging and reveals failures not captured by existing benchmarks. Our code and evaluation framework are publicly available at: https://github.com/xjtuleeyf/Locomo-Plus. |
| title | Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2602.10715 |