REMem: Reasoning with Episodic Memory in Language Agent

Fuente: arXiv
Saved in:
Bibliographic Details
Main Authors: Shu, Yiheng, Jonnalagedda, Saisri Padmaja, Gao, Xiang, Gutiérrez, Bernal Jiménez, Qi, Weijian, Das, Kamalika, Sun, Huan, Su, Yu
Format: Preprint
Published: 2026
Subjects:
Online Access:
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1866908858293682176
author Shu, Yiheng
Jonnalagedda, Saisri Padmaja
Gao, Xiang
Gutiérrez, Bernal Jiménez
Qi, Weijian
Das, Kamalika
Sun, Huan
Su, Yu
author_facet Shu, Yiheng
Jonnalagedda, Saisri Padmaja
Gao, Xiang
Gutiérrez, Bernal Jiménez
Qi, Weijian
Das, Kamalika
Sun, Huan
Su, Yu
contents Humans excel at remembering concrete experiences along spatiotemporal contexts and performing reasoning across those events, i.e., the capacity for episodic memory. In contrast, memory in language agents remains mainly semantic, and current agents are not yet capable of effectively recollecting and reasoning over interaction histories. We identify and formalize the core challenges of episodic recollection and reasoning from this gap, and observe that existing work often overlooks episodicity, lacks explicit event modeling, or overemphasizes simple retrieval rather than complex reasoning. We present REMem, a two-phase framework for constructing and reasoning with episodic memory: 1) Offline indexing, where REMem converts experiences into a hybrid memory graph that flexibly links time-aware gists and facts. 2) Online inference, where REMem employs an agentic retriever with carefully curated tools for iterative retrieval over the memory graph. Comprehensive evaluation across four episodic memory benchmarks shows that REMem substantially outperforms state-of-the-art memory systems such as Mem0 and HippoRAG 2, showing 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks, respectively. Moreover, REMem also demonstrates more robust refusal behavior for unanswerable questions.
format Preprint
id arxiv_https___arxiv_org_abs_2602_13530
institution arXiv
publishDate 2026
record_format arxiv
spellingShingle REMem: Reasoning with Episodic Memory in Language Agent
Shu, Yiheng
Jonnalagedda, Saisri Padmaja
Gao, Xiang
Gutiérrez, Bernal Jiménez
Qi, Weijian
Das, Kamalika
Sun, Huan
Su, Yu
Artificial Intelligence
Humans excel at remembering concrete experiences along spatiotemporal contexts and performing reasoning across those events, i.e., the capacity for episodic memory. In contrast, memory in language agents remains mainly semantic, and current agents are not yet capable of effectively recollecting and reasoning over interaction histories. We identify and formalize the core challenges of episodic recollection and reasoning from this gap, and observe that existing work often overlooks episodicity, lacks explicit event modeling, or overemphasizes simple retrieval rather than complex reasoning. We present REMem, a two-phase framework for constructing and reasoning with episodic memory: 1) Offline indexing, where REMem converts experiences into a hybrid memory graph that flexibly links time-aware gists and facts. 2) Online inference, where REMem employs an agentic retriever with carefully curated tools for iterative retrieval over the memory graph. Comprehensive evaluation across four episodic memory benchmarks shows that REMem substantially outperforms state-of-the-art memory systems such as Mem0 and HippoRAG 2, showing 3.4% and 13.4% absolute improvements on episodic recollection and reasoning tasks, respectively. Moreover, REMem also demonstrates more robust refusal behavior for unanswerable questions.
title REMem: Reasoning with Episodic Memory in Language Agent
topic Artificial Intelligence
url https://arxiv.org/abs/2602.13530