TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917241316966400 |
|---|---|
| author | Shen, Yuanzhe Huang, Zisu Wang, Zhengyuan Tian, Muzhao Guo, Zhengkang Zhang, Chenyang Zhou, Shuaiyu Hu, Zengjie Li, Dailin Xu, Jingwen Wang, Kaimin Liu, Wenhao Li, Tianlong Yue, Fengpeng Hong, Feng Liu, Cao Zeng, Ke |
| author_facet | Shen, Yuanzhe Huang, Zisu Wang, Zhengyuan Tian, Muzhao Guo, Zhengkang Zhang, Chenyang Zhou, Shuaiyu Hu, Zengjie Li, Dailin Xu, Jingwen Wang, Kaimin Liu, Wenhao Li, Tianlong Yue, Fengpeng Hong, Feng Liu, Cao Zeng, Ke |
| contents | As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce \textbf{TRIP-Bench}, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose \textbf{GTPO}, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01675 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios Shen, Yuanzhe Huang, Zisu Wang, Zhengyuan Tian, Muzhao Guo, Zhengkang Zhang, Chenyang Zhou, Shuaiyu Hu, Zengjie Li, Dailin Xu, Jingwen Wang, Kaimin Liu, Wenhao Li, Tianlong Yue, Fengpeng Hong, Feng Liu, Cao Zeng, Ke Artificial Intelligence Machine Learning As LLM-based agents are deployed in increasingly complex real-world settings, existing benchmarks underrepresent key challenges such as enforcing global constraints, coordinating multi-tool reasoning, and adapting to evolving user behavior over long, multi-turn interactions. To bridge this gap, we introduce \textbf{TRIP-Bench}, a long-horizon benchmark grounded in realistic travel-planning scenarios. TRIP-Bench leverages real-world data, offers 18 curated tools and 40+ travel requirements, and supports automated evaluation. It includes splits of varying difficulty; the hard split emphasizes long and ambiguous interactions, style shifts, feasibility changes, and iterative version revision. Dialogues span up to 15 user turns, can involve 150+ tool calls, and may exceed 200k tokens of context. Experiments show that even advanced models achieve at most 50\% success on the easy split, with performance dropping below 10\% on hard subsets. We further propose \textbf{GTPO}, an online multi-turn reinforcement learning method with specialized reward normalization and reward differencing. Applied to Qwen2.5-32B-Instruct, GTPO improves constraint satisfaction and interaction robustness, outperforming Gemini-3-Pro in our evaluation. We expect TRIP-Bench to advance practical long-horizon interactive agents, and GTPO to provide an effective online RL recipe for robust long-horizon training. |
| title | TRIP-Bench: A Benchmark for Long-Horizon Interactive Agents in Real-World Scenarios |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2602.01675 |