Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.16069 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910845084106752 |
|---|---|
| author | Kon, Patrick Tser Jern Liu, Jiachen Ding, Qiuyi Qiu, Yiming Yang, Zhenning Huang, Yibo Srinivasa, Jayanth Lee, Myungjin Chowdhury, Mosharaf Chen, Ang |
| author_facet | Kon, Patrick Tser Jern Liu, Jiachen Ding, Qiuyi Qiu, Yiming Yang, Zhenning Huang, Yibo Srinivasa, Jayanth Lee, Myungjin Chowdhury, Mosharaf Chen, Ang |
| contents | Scientific experimentation, a cornerstone of human progress, demands rigor in reliability, methodical control, and interpretability to yield meaningful results. Despite the growing capabilities of large language models (LLMs) in automating different aspects of the scientific process, automating rigorous experimentation remains a significant challenge. To address this gap, we propose Curie, an AI agent framework designed to embed rigor into the experimentation process through three key components: an intra-agent rigor module to enhance reliability, an inter-agent rigor module to maintain methodical control, and an experiment knowledge module to enhance interpretability. To evaluate Curie, we design a novel experimental benchmark composed of 46 questions across four computer science domains, derived from influential research papers, and widely adopted open-source projects. Compared to the strongest baseline tested, we achieve a 3.4$\times$ improvement in correctly answering experimental questions. Curie is open-sourced at https://github.com/Just-Curieous/Curie. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_16069 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents Kon, Patrick Tser Jern Liu, Jiachen Ding, Qiuyi Qiu, Yiming Yang, Zhenning Huang, Yibo Srinivasa, Jayanth Lee, Myungjin Chowdhury, Mosharaf Chen, Ang Artificial Intelligence Machine Learning Scientific experimentation, a cornerstone of human progress, demands rigor in reliability, methodical control, and interpretability to yield meaningful results. Despite the growing capabilities of large language models (LLMs) in automating different aspects of the scientific process, automating rigorous experimentation remains a significant challenge. To address this gap, we propose Curie, an AI agent framework designed to embed rigor into the experimentation process through three key components: an intra-agent rigor module to enhance reliability, an inter-agent rigor module to maintain methodical control, and an experiment knowledge module to enhance interpretability. To evaluate Curie, we design a novel experimental benchmark composed of 46 questions across four computer science domains, derived from influential research papers, and widely adopted open-source projects. Compared to the strongest baseline tested, we achieve a 3.4$\times$ improvement in correctly answering experimental questions. Curie is open-sourced at https://github.com/Just-Curieous/Curie. |
| title | Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents |
| topic | Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2502.16069 |