Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2506.15741 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918067273990144 |
|---|---|
| author | Zhu, He Qin, Tianrui Zhu, King Huang, Heyuan Guan, Yeyi Xia, Jinxiang Yao, Yi Li, Hanhao Wang, Ningning Liu, Pai Peng, Tianhao Gui, Xin Li, Xiaowan Liu, Yuhui Jiang, Yuchen Eleanor Wang, Jun Zhang, Changwang Tang, Xiangru Zhang, Ge Yang, Jian Liu, Minghao Gao, Xitong Liu, Jiaheng Zhou, Wangchunshu |
| author_facet | Zhu, He Qin, Tianrui Zhu, King Huang, Heyuan Guan, Yeyi Xia, Jinxiang Yao, Yi Li, Hanhao Wang, Ningning Liu, Pai Peng, Tianhao Gui, Xin Li, Xiaowan Liu, Yuhui Jiang, Yuchen Eleanor Wang, Jun Zhang, Changwang Tang, Xiangru Zhang, Ge Yang, Jian Liu, Minghao Gao, Xitong Liu, Jiaheng Zhou, Wangchunshu |
| contents | Recently, Agentic AI has become an increasingly popular research field. However, we argue that current agent research practices lack standardization and scientific rigor, making it hard to conduct fair comparisons among methods. As a result, it is still unclear how different design choices in agent frameworks affect effectiveness, and measuring their progress remains challenging. In this work, we conduct a systematic empirical study on GAIA benchmark and BrowseComp to examine the impact of popular design choices in key agent components in a fair and rigorous manner. We find that the lack of a standard evaluation protocol makes previous works, even open-sourced ones, non-reproducible, with significant variance between random runs. Therefore, we introduce a more robust evaluation protocol to stabilize comparisons. Our study reveals which components and designs are crucial for effective agents, while others are redundant, despite seeming logical. Based on our findings, we build and open-source OAgents, a new foundation agent framework that achieves state-of-the-art performance among open-source projects. OAgents offers a modular design for various agent components, promoting future research in Agentic AI. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15741 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OAgents: An Empirical Study of Building Effective Agents Zhu, He Qin, Tianrui Zhu, King Huang, Heyuan Guan, Yeyi Xia, Jinxiang Yao, Yi Li, Hanhao Wang, Ningning Liu, Pai Peng, Tianhao Gui, Xin Li, Xiaowan Liu, Yuhui Jiang, Yuchen Eleanor Wang, Jun Zhang, Changwang Tang, Xiangru Zhang, Ge Yang, Jian Liu, Minghao Gao, Xitong Liu, Jiaheng Zhou, Wangchunshu Artificial Intelligence Computation and Language Recently, Agentic AI has become an increasingly popular research field. However, we argue that current agent research practices lack standardization and scientific rigor, making it hard to conduct fair comparisons among methods. As a result, it is still unclear how different design choices in agent frameworks affect effectiveness, and measuring their progress remains challenging. In this work, we conduct a systematic empirical study on GAIA benchmark and BrowseComp to examine the impact of popular design choices in key agent components in a fair and rigorous manner. We find that the lack of a standard evaluation protocol makes previous works, even open-sourced ones, non-reproducible, with significant variance between random runs. Therefore, we introduce a more robust evaluation protocol to stabilize comparisons. Our study reveals which components and designs are crucial for effective agents, while others are redundant, despite seeming logical. Based on our findings, we build and open-source OAgents, a new foundation agent framework that achieves state-of-the-art performance among open-source projects. OAgents offers a modular design for various agent components, promoting future research in Agentic AI. |
| title | OAgents: An Empirical Study of Building Effective Agents |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2506.15741 |