Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914238062133248 |
|---|---|
| author | Bei, Yuanchen Wei, Tianxin Ning, Xuying Zhao, Yanjun Liu, Zhining Lin, Xiao Zhu, Yada Hamann, Hendrik He, Jingrui Tong, Hanghang |
| author_facet | Bei, Yuanchen Wei, Tianxin Ning, Xuying Zhao, Yanjun Liu, Zhining Lin, Xiao Zhu, Yada Hamann, Hendrik He, Jingrui Tong, Hanghang |
| contents | Long-term memory is a critical capability for multimodal large language model (MLLM) agents, particularly in conversational settings where information accumulates and evolves over time. However, existing benchmarks either evaluate multi-session memory in text-only conversations or assess multimodal understanding within localized contexts, failing to evaluate how multimodal memory is preserved, organized, and evolved across long-term conversational trajectories. Thus, we introduce Mem-Gallery, a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents. Mem-Gallery features high-quality multi-session conversations grounded in both visual and textual information, with long interaction horizons and rich multimodal dependencies. Building on this dataset, we propose a systematic evaluation framework that assesses key memory capabilities along three functional dimensions: memory extraction and test-time adaptation, memory reasoning, and memory knowledge management. Extensive benchmarking across thirteen memory systems reveals several key findings, highlighting the necessity of explicit multimodal information retention and memory organization, the persistent limitations in memory reasoning and knowledge management, as well as the efficiency bottleneck of current models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_03515 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents Bei, Yuanchen Wei, Tianxin Ning, Xuying Zhao, Yanjun Liu, Zhining Lin, Xiao Zhu, Yada Hamann, Hendrik He, Jingrui Tong, Hanghang Computation and Language Artificial Intelligence Long-term memory is a critical capability for multimodal large language model (MLLM) agents, particularly in conversational settings where information accumulates and evolves over time. However, existing benchmarks either evaluate multi-session memory in text-only conversations or assess multimodal understanding within localized contexts, failing to evaluate how multimodal memory is preserved, organized, and evolved across long-term conversational trajectories. Thus, we introduce Mem-Gallery, a new benchmark for evaluating multimodal long-term conversational memory in MLLM agents. Mem-Gallery features high-quality multi-session conversations grounded in both visual and textual information, with long interaction horizons and rich multimodal dependencies. Building on this dataset, we propose a systematic evaluation framework that assesses key memory capabilities along three functional dimensions: memory extraction and test-time adaptation, memory reasoning, and memory knowledge management. Extensive benchmarking across thirteen memory systems reveals several key findings, highlighting the necessity of explicit multimodal information retention and memory organization, the persistent limitations in memory reasoning and knowledge management, as well as the efficiency bottleneck of current models. |
| title | Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2601.03515 |