Cover Image

SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yiheng, Chen, Yixin, Li, Shuo, Zhou, Yifan, Liu, Bo, Gao, Hengjian, Yuan, Jiakang, Bu, Jia, Xu, Wanghan, Zhou, Yuhao, Zhao, Xiangyu, Zhou, Zhiwang, Wang, Fengxiang, Duan, Haodong, Zhang, Songyang, Yao, Jun, Deng, Han, Wang, Yizhou, Xiao, Jiabei, Liu, Jiaqi, Su, Encheng, Liu, Yujie, Wang, Weida, Yao, Junchi, Zheng, Shenghe, Sun, Haoran, Ma, Runmin, Yan, Xiangchao, Zhang, Bo, Zhou, Dongzhan, Zhang, Shufei, Ye, Peng, Wang, Xiaosong, Tang, Shixiang, Zhang, Wenlong, Bai, Lei
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866917185594589184
author	Wang, Yiheng Chen, Yixin Li, Shuo Zhou, Yifan Liu, Bo Gao, Hengjian Yuan, Jiakang Bu, Jia Xu, Wanghan Zhou, Yuhao Zhao, Xiangyu Zhou, Zhiwang Wang, Fengxiang Duan, Haodong Zhang, Songyang Yao, Jun Deng, Han Wang, Yizhou Xiao, Jiabei Liu, Jiaqi Su, Encheng Liu, Yujie Wang, Weida Yao, Junchi Zheng, Shenghe Sun, Haoran Ma, Runmin Yan, Xiangchao Zhang, Bo Zhou, Dongzhan Zhang, Shufei Ye, Peng Wang, Xiaosong Tang, Shixiang Zhang, Wenlong Bai, Lei
author_facet	Wang, Yiheng Chen, Yixin Li, Shuo Zhou, Yifan Liu, Bo Gao, Hengjian Yuan, Jiakang Bu, Jia Xu, Wanghan Zhou, Yuhao Zhao, Xiangyu Zhou, Zhiwang Wang, Fengxiang Duan, Haodong Zhang, Songyang Yao, Jun Deng, Han Wang, Yizhou Xiao, Jiabei Liu, Jiaqi Su, Encheng Liu, Yujie Wang, Weida Yao, Junchi Zheng, Shenghe Sun, Haoran Ma, Runmin Yan, Xiangchao Zhang, Bo Zhou, Dongzhan Zhang, Shufei Ye, Peng Wang, Xiaosong Tang, Shixiang Zhang, Wenlong Bai, Lei
contents	We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
format	Preprint
id	arxiv_https___arxiv_org_abs_2512_22334
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence Wang, Yiheng Chen, Yixin Li, Shuo Zhou, Yifan Liu, Bo Gao, Hengjian Yuan, Jiakang Bu, Jia Xu, Wanghan Zhou, Yuhao Zhao, Xiangyu Zhou, Zhiwang Wang, Fengxiang Duan, Haodong Zhang, Songyang Yao, Jun Deng, Han Wang, Yizhou Xiao, Jiabei Liu, Jiaqi Su, Encheng Liu, Yujie Wang, Weida Yao, Junchi Zheng, Shenghe Sun, Haoran Ma, Runmin Yan, Xiangchao Zhang, Bo Zhou, Dongzhan Zhang, Shufei Ye, Peng Wang, Xiaosong Tang, Shixiang Zhang, Wenlong Bai, Lei Artificial Intelligence Computation and Language We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
title	SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence
topic	Artificial Intelligence Computation and Language
url	https://arxiv.org/abs/2512.22334