Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.30842 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910271654592512 |
|---|---|
| author | Zhang, Yuwei Dong, Chengyu Jin, Shuowei Yu, Changlong Cui, Hejie Jin, Hongye Zhang, Xinyang Bonab, Hamed Lockard, Colin Chen, Jianshu Shi, Zhenyu Shang, Jingbo Li, Xian Yin, Bing |
| author_facet | Zhang, Yuwei Dong, Chengyu Jin, Shuowei Yu, Changlong Cui, Hejie Jin, Hongye Zhang, Xinyang Bonab, Hamed Lockard, Colin Chen, Jianshu Shi, Zhenyu Shang, Jingbo Li, Xian Yin, Bing |
| contents | Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs substantial decoding overhead for the extra summarization tokens, which significantly affect the end-to-end response latency at deployment. In this paper, we introduce CoMem, a novel framework that decouples memory management from the primary agent workflow, enabling these processes to execute in parallel. We propose a $k$-step-off asynchronous pipeline that overlaps the memory model's summarization with the agent's inference, effectively masking the latency of context processing. To ensure robustness under this asynchronous setting, we introduce a reward-driven training strategy that aligns the memory model to capture sufficient statistics for the agent's decision-making. Theoretical analysis confirms that CoMem offers a superior efficiency-effectiveness trade-off compared to coupled architectures. Our extensive experimental results on SWE-Bench-Verified show that CoMem provides 1.4x latency improvements upon vanilla long-context solutions while preserving most of the performance. Furthermore, we demonstrate that these latency gains scale favorably with increased system throughput, offering a modular path forward for the independent optimization of agent reasoning and memory compression. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_30842 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | CoMem: Context Management with A Decoupled Long-Context Model Zhang, Yuwei Dong, Chengyu Jin, Shuowei Yu, Changlong Cui, Hejie Jin, Hongye Zhang, Xinyang Bonab, Hamed Lockard, Colin Chen, Jianshu Shi, Zhenyu Shang, Jingbo Li, Xian Yin, Bing Machine Learning Context management enables agentic models to solve long-horizon tasks through iterative summarization of previous interaction histories. However, this process typically incurs substantial decoding overhead for the extra summarization tokens, which significantly affect the end-to-end response latency at deployment. In this paper, we introduce CoMem, a novel framework that decouples memory management from the primary agent workflow, enabling these processes to execute in parallel. We propose a $k$-step-off asynchronous pipeline that overlaps the memory model's summarization with the agent's inference, effectively masking the latency of context processing. To ensure robustness under this asynchronous setting, we introduce a reward-driven training strategy that aligns the memory model to capture sufficient statistics for the agent's decision-making. Theoretical analysis confirms that CoMem offers a superior efficiency-effectiveness trade-off compared to coupled architectures. Our extensive experimental results on SWE-Bench-Verified show that CoMem provides 1.4x latency improvements upon vanilla long-context solutions while preserving most of the performance. Furthermore, we demonstrate that these latency gains scale favorably with increased system throughput, offering a modular path forward for the independent optimization of agent reasoning and memory compression. |
| title | CoMem: Context Management with A Decoupled Long-Context Model |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.30842 |