AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910262583361536 |
|---|---|
| author | Zhao, Yujie Yuan, Boqin Huang, Junbo Yuan, Haocheng Yu, Zhongming Xu, Haozhou Hu, Lanxiang Shankarampeta, Abhilash Huang, Zimeng Ni, Wentao Tian, Yuandong Zhao, Jishen |
| author_facet | Zhao, Yujie Yuan, Boqin Huang, Junbo Yuan, Haocheng Yu, Zhongming Xu, Haozhou Hu, Lanxiang Shankarampeta, Abhilash Huang, Zimeng Ni, Wentao Tian, Yuandong Zhao, Jishen |
| contents | Large Language Models (LLMs) are increasingly used as autonomous agents in complex, long-horizon applications, where effective memory is critical for sustained performance. Yet existing memory benchmarks are largely dialogue-centric, while real agent memory consists of continuous agent-environment interaction trajectories composed of states, actions, observations, and tool outputs. To address this gap, we introduce **AMA-Bench** (**A**gent **M**emory with **A**ny length), a benchmark for evaluating long-horizon memory in realistic agentic settings. AMA-Bench combines real-world agent trajectories from representative applications with expert-curated QA, as well as synthetic trajectories that scale to arbitrary horizons with rule-based QA. Our study shows that existing memory systems underperform because they fail to capture causal and objective information and rely heavily on lossy similarity-based retrieval. We further propose **AMA-Agent**, a memory system based on causality-graph construction and tool-augmented retrieval. AMA-Agent achieves **57.22%** accuracy on AMA-Bench, outperforming the strongest baseline by **11.16%**. Resources are available at: [https://ama-bench.github.io/](https://ama-bench.github.io/). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_22769 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications Zhao, Yujie Yuan, Boqin Huang, Junbo Yuan, Haocheng Yu, Zhongming Xu, Haozhou Hu, Lanxiang Shankarampeta, Abhilash Huang, Zimeng Ni, Wentao Tian, Yuandong Zhao, Jishen Artificial Intelligence Machine Learning Large Language Models (LLMs) are increasingly used as autonomous agents in complex, long-horizon applications, where effective memory is critical for sustained performance. Yet existing memory benchmarks are largely dialogue-centric, while real agent memory consists of continuous agent-environment interaction trajectories composed of states, actions, observations, and tool outputs. To address this gap, we introduce **AMA-Bench** (**A**gent **M**emory with **A**ny length), a benchmark for evaluating long-horizon memory in realistic agentic settings. AMA-Bench combines real-world agent trajectories from representative applications with expert-curated QA, as well as synthetic trajectories that scale to arbitrary horizons with rule-based QA. Our study shows that existing memory systems underperform because they fail to capture causal and objective information and rely heavily on lossy similarity-based retrieval. We further propose **AMA-Agent**, a memory system based on causality-graph construction and tool-augmented retrieval. AMA-Agent achieves **57.22%** accuracy on AMA-Bench, outperforming the strongest baseline by **11.16%**. Resources are available at: [https://ama-bench.github.io/](https://ama-bench.github.io/). |
| title | AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.22769 |