Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912716623446016 |
|---|---|
| author | Gandhi, Kahaan Bolliet, Boris Zubeldia, Inigo |
| author_facet | Gandhi, Kahaan Bolliet, Boris Zubeldia, Inigo |
| contents | We show that multi-agent systems guided by vision-language models (VLMs) improve end-to-end autonomous scientific discovery. By treating plots as verifiable checkpoints, a VLM-as-a-judge evaluates figures against dynamically generated domain-specific rubrics, enabling agents to correct their own errors and steer exploratory data analysis in real-time. Case studies in cosmology and astrochemistry demonstrate recovery from faulty reasoning paths and adaptation to new datasets without human intervention. On a 10-task benchmark for data-driven discovery, VLM-augmented systems achieve pass at 1 scores of 0.7-0.8, compared to 0.2-0.3 for code-only and 0.4-0.5 for code-and-text baselines, while also providing auditable reasoning traces that improve interpretability. Code available here: https://github.com/CMBAgents/cmbagent |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_14631 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities Gandhi, Kahaan Bolliet, Boris Zubeldia, Inigo Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Multiagent Systems We show that multi-agent systems guided by vision-language models (VLMs) improve end-to-end autonomous scientific discovery. By treating plots as verifiable checkpoints, a VLM-as-a-judge evaluates figures against dynamically generated domain-specific rubrics, enabling agents to correct their own errors and steer exploratory data analysis in real-time. Case studies in cosmology and astrochemistry demonstrate recovery from faulty reasoning paths and adaptation to new datasets without human intervention. On a 10-task benchmark for data-driven discovery, VLM-augmented systems achieve pass at 1 scores of 0.7-0.8, compared to 0.2-0.3 for code-only and 0.4-0.5 for code-and-text baselines, while also providing auditable reasoning traces that improve interpretability. Code available here: https://github.com/CMBAgents/cmbagent |
| title | Enhancing Agentic Autonomous Scientific Discovery with Vision-Language Model Capabilities |
| topic | Computation and Language Artificial Intelligence Computer Vision and Pattern Recognition Multiagent Systems |
| url | https://arxiv.org/abs/2511.14631 |