BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914539601133568 |
|---|---|
| author | Wu, Jinge Zhou, Hongjian Zeng, Mingde Zhu, Jiayuan Wu, Junde Pan, Jiazhen Wu, Sean Wu, Honghan Liu, Fenglin Clifton, David A. |
| author_facet | Wu, Jinge Zhou, Hongjian Zeng, Mingde Zhu, Jiayuan Wu, Junde Pan, Jiazhen Wu, Sean Wu, Honghan Liu, Fenglin Clifton, David A. |
| contents | Building a deep research agent today is an exercise in glue code: the same backbone evaluated on the same benchmark can report different accuracies in different papers because harness and tool registry all differ, and integrating a new foundation model into a comparable evaluation surface costs weeks of model-specific engineering. We call this the per-paper engineering tax and release BioMedArena, an open-source toolkit that not only alleviates it but also provides an arena for fair comparison of different foundation models when evaluating them as deep-research agents. BioMedArena decouples six layers of biomedical agent evaluation -- benchmark loading, tool exposure, tool selection, execution mode, context management, and scoring -- and exposes 147 biomedical benchmarks and 75 biomedical tools across 9 functional families. Adding a new model, benchmark, or tool reduces to registering a few-line provider adapter. We further provide 6 agent harnesses with 6 context-management strategies, which provide 12 backbones with competitive research capabilities and significantly improved performance, achieving state-of-the-art (SOTA) results on 8 representative biomedical benchmarks, with an average lift of +15.03 percentage points over prior SOTA. The toolkit, configurations, and per-task traces are available at https://github.com/AI-in-Health/BioMedArena |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_06177 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents Wu, Jinge Zhou, Hongjian Zeng, Mingde Zhu, Jiayuan Wu, Junde Pan, Jiazhen Wu, Sean Wu, Honghan Liu, Fenglin Clifton, David A. Artificial Intelligence Building a deep research agent today is an exercise in glue code: the same backbone evaluated on the same benchmark can report different accuracies in different papers because harness and tool registry all differ, and integrating a new foundation model into a comparable evaluation surface costs weeks of model-specific engineering. We call this the per-paper engineering tax and release BioMedArena, an open-source toolkit that not only alleviates it but also provides an arena for fair comparison of different foundation models when evaluating them as deep-research agents. BioMedArena decouples six layers of biomedical agent evaluation -- benchmark loading, tool exposure, tool selection, execution mode, context management, and scoring -- and exposes 147 biomedical benchmarks and 75 biomedical tools across 9 functional families. Adding a new model, benchmark, or tool reduces to registering a few-line provider adapter. We further provide 6 agent harnesses with 6 context-management strategies, which provide 12 backbones with competitive research capabilities and significantly improved performance, achieving state-of-the-art (SOTA) results on 8 representative biomedical benchmarks, with an average lift of +15.03 percentage points over prior SOTA. The toolkit, configurations, and per-task traces are available at https://github.com/AI-in-Health/BioMedArena |
| title | BioMedArena: An Open-source Toolkit for Building and Evaluating Biomedical Deep Research Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2605.06177 |