MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915558762479616 |
|---|---|
| author | Lu, Yuxing Zhao, Xukai Tamo, J. Ben Nnamdi, Micky C. Peng, Rui Zeng, Shuang Hu, Xingyu Wang, Jinzhuo Wang, May D. |
| author_facet | Lu, Yuxing Zhao, Xukai Tamo, J. Ben Nnamdi, Micky C. Peng, Rui Zeng, Shuang Hu, Xingyu Wang, Jinzhuo Wang, May D. |
| contents | Large Language Models (LLMs) have demonstrated remarkable capabilities on general text; however, their proficiency in specialized scientific domains that require deep, interconnected knowledge remains largely uncharacterized. Metabolomics presents unique challenges with its complex biochemical pathways, heterogeneous identifier systems, and fragmented databases. To systematically evaluate LLM capabilities in this domain, we introduce MetaBench, the first benchmark for metabolomics assessment. Curated from authoritative public resources, MetaBench evaluates five capabilities essential for metabolomics research: knowledge, understanding, grounding, reasoning, and research. Our evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks: while models perform well on text generation tasks, cross-database identifier grounding remains challenging even with retrieval augmentation. Model performance also decreases on long-tail metabolites with sparse annotations. With MetaBench, we provide essential infrastructure for developing and evaluating metabolomics AI systems, enabling systematic progress toward reliable computational tools for metabolomics research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_14944 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics Lu, Yuxing Zhao, Xukai Tamo, J. Ben Nnamdi, Micky C. Peng, Rui Zeng, Shuang Hu, Xingyu Wang, Jinzhuo Wang, May D. Computation and Language Artificial Intelligence Computational Engineering, Finance, and Science Large Language Models (LLMs) have demonstrated remarkable capabilities on general text; however, their proficiency in specialized scientific domains that require deep, interconnected knowledge remains largely uncharacterized. Metabolomics presents unique challenges with its complex biochemical pathways, heterogeneous identifier systems, and fragmented databases. To systematically evaluate LLM capabilities in this domain, we introduce MetaBench, the first benchmark for metabolomics assessment. Curated from authoritative public resources, MetaBench evaluates five capabilities essential for metabolomics research: knowledge, understanding, grounding, reasoning, and research. Our evaluation of 25 open- and closed-source LLMs reveals distinct performance patterns across metabolomics tasks: while models perform well on text generation tasks, cross-database identifier grounding remains challenging even with retrieval augmentation. Model performance also decreases on long-tail metabolites with sparse annotations. With MetaBench, we provide essential infrastructure for developing and evaluating metabolomics AI systems, enabling systematic progress toward reliable computational tools for metabolomics research. |
| title | MetaBench: A Multi-task Benchmark for Assessing LLMs in Metabolomics |
| topic | Computation and Language Artificial Intelligence Computational Engineering, Finance, and Science |
| url | https://arxiv.org/abs/2510.14944 |