HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866917438470225920 |
|---|---|
| author | Hong, Hanhua LI, Yizhi Chen, Jiaoyan Ananiadou, Sophia Li, Xiaoli Kim, Jung-jae Lin, Chenghua |
| author_facet | Hong, Hanhua LI, Yizhi Chen, Jiaoyan Ananiadou, Sophia Li, Xiaoli Kim, Jung-jae Lin, Chenghua |
| contents | Recent advances in large language models have highlighted their potential to automate computational research, particularly reproducing experimental results. However, existing approaches still use fixed sequential agent pipelines with weak global coordination, which limits their robustness and overall performance. In this work, we propose Hierarchical Research Agent System (HiRAS), a hierarchical multi-agent framework for end-to-end experiment reproduction that employs supervisory manager agents to coordinate specialised agents across fine-grained stages. We also identify limitations in the reference-free evaluation of the Paper2Code benchmark and introduce Paper2Code-Extra (P2C-Ex), a refined protocol that incorporates repository-level information and better aligns with the original reference-based metric. We conduct extensive evaluation, validating the effectiveness and robustness of our proposed methods, and observing improvements, including >10\% relative performance gain beyond the previous state-of-the-art using open-source backbone models and significantly reduced hallucination in evaluation. Our work is available on GitHub: https://github.com/KOU-199024/HiRAS. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_17745 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution Hong, Hanhua LI, Yizhi Chen, Jiaoyan Ananiadou, Sophia Li, Xiaoli Kim, Jung-jae Lin, Chenghua Computation and Language Recent advances in large language models have highlighted their potential to automate computational research, particularly reproducing experimental results. However, existing approaches still use fixed sequential agent pipelines with weak global coordination, which limits their robustness and overall performance. In this work, we propose Hierarchical Research Agent System (HiRAS), a hierarchical multi-agent framework for end-to-end experiment reproduction that employs supervisory manager agents to coordinate specialised agents across fine-grained stages. We also identify limitations in the reference-free evaluation of the Paper2Code benchmark and introduce Paper2Code-Extra (P2C-Ex), a refined protocol that incorporates repository-level information and better aligns with the original reference-based metric. We conduct extensive evaluation, validating the effectiveness and robustness of our proposed methods, and observing improvements, including >10\% relative performance gain beyond the previous state-of-the-art using open-source backbone models and significantly reduced hallucination in evaluation. Our work is available on GitHub: https://github.com/KOU-199024/HiRAS. |
| title | HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2604.17745 |