MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916824181899264 |
|---|---|
| author | Yu, Hongli Chen, Tinghong Feng, Jiangtao Chen, Jiangjie Dai, Weinan Yu, Qiying Zhang, Ya-Qin Ma, Wei-Ying Liu, Jingjing Wang, Mingxuan Zhou, Hao |
| author_facet | Yu, Hongli Chen, Tinghong Feng, Jiangtao Chen, Jiangjie Dai, Weinan Yu, Qiying Zhang, Ya-Qin Ma, Wei-Ying Liu, Jingjing Wang, Mingxuan Zhou, Hao |
| contents | Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text processing. We directly optimize for long-text tasks in an end-to-end fashion and introduce a novel agent workflow, MemAgent, which reads text in segments and updates the memory using an overwrite strategy. We extend the DAPO algorithm to facilitate training via independent-context multi-conversation generation. MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss < 5% and achieves 95%+ in 512K RULER test. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02259 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent Yu, Hongli Chen, Tinghong Feng, Jiangtao Chen, Jiangjie Dai, Weinan Yu, Qiying Zhang, Ya-Qin Ma, Wei-Ying Liu, Jingjing Wang, Mingxuan Zhou, Hao Computation and Language Artificial Intelligence Machine Learning Despite improvements by length extrapolation, efficient attention and memory modules, handling infinitely long documents with linear complexity without performance degradation during extrapolation remains the ultimate challenge in long-text processing. We directly optimize for long-text tasks in an end-to-end fashion and introduce a novel agent workflow, MemAgent, which reads text in segments and updates the memory using an overwrite strategy. We extend the DAPO algorithm to facilitate training via independent-context multi-conversation generation. MemAgent has demonstrated superb long-context capabilities, being able to extrapolate from an 8K context trained on 32K text to a 3.5M QA task with performance loss < 5% and achieves 95%+ in 512K RULER test. |
| title | MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2507.02259 |