Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909988790730752 |
|---|---|
| author | Ma, Weitao Feng, Xiaocheng Huang, Lei Feng, Xiachong Ma, Zhanyu Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Qin, Bing |
| author_facet | Ma, Weitao Feng, Xiaocheng Huang, Lei Feng, Xiachong Ma, Zhanyu Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Qin, Bing |
| contents | Effective memory management is essential for large language model agents to navigate long-horizon tasks. Recent research has explored using Reinforcement Learning to develop specialized memory manager agents. However, existing approaches rely on final task performance as the primary reward, which results in severe reward sparsity and ineffective credit assignment, providing insufficient guidance for individual memory operations. To this end, we propose Fine-Mem, a unified framework designed for fine-grained feedback alignment. First, we introduce a Chunk-level Step Reward to provide immediate step-level supervision via auxiliary chunk-specific question answering tasks. Second, we devise Evidence-Anchored Reward Attribution to redistribute global rewards by anchoring credit to key memory operations, based on the specific memory items utilized as evidence in reasoning. Together, these components enable stable policy optimization and align local memory operations with the long-term utility of memory. Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines, achieving superior success rates across various sub-tasks. Further analysis reveals its adaptability and strong generalization capabilities across diverse model configurations and backbones. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_08435 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management Ma, Weitao Feng, Xiaocheng Huang, Lei Feng, Xiachong Ma, Zhanyu Xu, Jun Gao, Jiuchong Hao, Jinghua He, Renqing Qin, Bing Computation and Language Effective memory management is essential for large language model agents to navigate long-horizon tasks. Recent research has explored using Reinforcement Learning to develop specialized memory manager agents. However, existing approaches rely on final task performance as the primary reward, which results in severe reward sparsity and ineffective credit assignment, providing insufficient guidance for individual memory operations. To this end, we propose Fine-Mem, a unified framework designed for fine-grained feedback alignment. First, we introduce a Chunk-level Step Reward to provide immediate step-level supervision via auxiliary chunk-specific question answering tasks. Second, we devise Evidence-Anchored Reward Attribution to redistribute global rewards by anchoring credit to key memory operations, based on the specific memory items utilized as evidence in reasoning. Together, these components enable stable policy optimization and align local memory operations with the long-term utility of memory. Experiments on Memalpha and MemoryAgentBench demonstrate that Fine-Mem consistently outperforms strong baselines, achieving superior success rates across various sub-tasks. Further analysis reveals its adaptability and strong generalization capabilities across diverse model configurations and backbones. |
| title | Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.08435 |