Copertina

Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Xu, Wanghan, Zhou, Yuhao, Zhou, Yifan, Cao, Qinglong, Li, Shuo, Bu, Jia, Liu, Bo, Chen, Yixin, He, Xuming, Zhao, Xiangyu, Zhuang, Xiang, Wang, Fengxiang, Zhou, Zhiwang, Feng, Qiantai, Huang, Wenxuan, Wei, Jiaqi, Wu, Hao, Yang, Yuejin, Wang, Guangshuai, Xu, Sheng, Huang, Ziyan, Liu, Xinyao, Liu, Jiyao, Tang, Cheng, Li, Wei, Chen, Ying, Ning, Junzhi, Jiang, Pengfei, Ma, Chenglong, Du, Ye, Ji, Changkai, Xu, Huihui, Hu, Ming, Zheng, Jiangbin, Chen, Xin, Wu, Yucheng, Jiang, Feifei, Chen, Xi, Tang, Xiangru, Fu, Yuchen, Lu, Yingzhou, Zhang, Yuanyuan, Sun, Lihao, Li, Chengbo, Ma, Jinzhe, Liu, Wanhao, Liu, Yating, Wu, Kuo-Cheng, Chai, Shengdu, Wang, Yizhou, Zhangjin, Ouwen, Tang, Chen, Zhang, Shufei, Cao, Wenbo, Ren, Junjie, Cui, Taoyong, Yao, Zhouheng, Deng, Juntao, Sun, Yijie, Liu, Feng, Wei, Wangxu, Xu, Jingyi, Li, Zhangrui, Gong, Junchao, Guo, Zijie, Yao, Zhiyu, Chen, Zaoyu, Peng, Tianhao, Yu, Fangchen, Zhang, Bo, Zhou, Dongzhan, Tang, Shixiang, Liu, Jiaheng, Ling, Fenghua, Lu, Yan, Ren, Yuchen, Fei, Ben, Zhao, Zhen, Gu, Xinyu, Su, Rui, Wu, Xiao-Ming, Si, Weikang, Liu, Yang, Chen, Hao, Yan, Xiangchao, Yang, Xue, Yan, Junchi, Wu, Jiamin, Zheng, Qihao, Li, Chenhui, Gao, Zhiqiang, Kong, Hao, He, Junjun, Su, Mao, Fu, Tianfan, Ye, Peng, Song, Chunfeng, Dong, Nanqing, Li, Yuqiang, Fu, Huazhu, Sun, Siqi, Cheng, Lijing, Lin, Jintai, Ouyang, Wanli, Zhou, Bowen, Zhang, Wenlong, Bai, Lei
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Artificial Intelligence Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

_version_	1866908721518477312
author	Xu, Wanghan Zhou, Yuhao Zhou, Yifan Cao, Qinglong Li, Shuo Bu, Jia Liu, Bo Chen, Yixin He, Xuming Zhao, Xiangyu Zhuang, Xiang Wang, Fengxiang Zhou, Zhiwang Feng, Qiantai Huang, Wenxuan Wei, Jiaqi Wu, Hao Yang, Yuejin Wang, Guangshuai Xu, Sheng Huang, Ziyan Liu, Xinyao Liu, Jiyao Tang, Cheng Li, Wei Chen, Ying Ning, Junzhi Jiang, Pengfei Ma, Chenglong Du, Ye Ji, Changkai Xu, Huihui Hu, Ming Zheng, Jiangbin Chen, Xin Wu, Yucheng Jiang, Feifei Chen, Xi Tang, Xiangru Fu, Yuchen Lu, Yingzhou Zhang, Yuanyuan Sun, Lihao Li, Chengbo Ma, Jinzhe Liu, Wanhao Liu, Yating Wu, Kuo-Cheng Chai, Shengdu Wang, Yizhou Zhangjin, Ouwen Tang, Chen Zhang, Shufei Cao, Wenbo Ren, Junjie Cui, Taoyong Yao, Zhouheng Deng, Juntao Sun, Yijie Liu, Feng Wei, Wangxu Xu, Jingyi Li, Zhangrui Gong, Junchao Guo, Zijie Yao, Zhiyu Chen, Zaoyu Peng, Tianhao Yu, Fangchen Zhang, Bo Zhou, Dongzhan Tang, Shixiang Liu, Jiaheng Ling, Fenghua Lu, Yan Ren, Yuchen Fei, Ben Zhao, Zhen Gu, Xinyu Su, Rui Wu, Xiao-Ming Si, Weikang Liu, Yang Chen, Hao Yan, Xiangchao Yang, Xue Yan, Junchi Wu, Jiamin Zheng, Qihao Li, Chenhui Gao, Zhiqiang Kong, Hao He, Junjun Su, Mao Fu, Tianfan Ye, Peng Song, Chunfeng Dong, Nanqing Li, Yuqiang Fu, Huazhu Sun, Siqi Cheng, Lijing Lin, Jintai Ouyang, Wanli Zhou, Bowen Zhang, Wenlong Bai, Lei
author_facet	Xu, Wanghan Zhou, Yuhao Zhou, Yifan Cao, Qinglong Li, Shuo Bu, Jia Liu, Bo Chen, Yixin He, Xuming Zhao, Xiangyu Zhuang, Xiang Wang, Fengxiang Zhou, Zhiwang Feng, Qiantai Huang, Wenxuan Wei, Jiaqi Wu, Hao Yang, Yuejin Wang, Guangshuai Xu, Sheng Huang, Ziyan Liu, Xinyao Liu, Jiyao Tang, Cheng Li, Wei Chen, Ying Ning, Junzhi Jiang, Pengfei Ma, Chenglong Du, Ye Ji, Changkai Xu, Huihui Hu, Ming Zheng, Jiangbin Chen, Xin Wu, Yucheng Jiang, Feifei Chen, Xi Tang, Xiangru Fu, Yuchen Lu, Yingzhou Zhang, Yuanyuan Sun, Lihao Li, Chengbo Ma, Jinzhe Liu, Wanhao Liu, Yating Wu, Kuo-Cheng Chai, Shengdu Wang, Yizhou Zhangjin, Ouwen Tang, Chen Zhang, Shufei Cao, Wenbo Ren, Junjie Cui, Taoyong Yao, Zhouheng Deng, Juntao Sun, Yijie Liu, Feng Wei, Wangxu Xu, Jingyi Li, Zhangrui Gong, Junchao Guo, Zijie Yao, Zhiyu Chen, Zaoyu Peng, Tianhao Yu, Fangchen Zhang, Bo Zhou, Dongzhan Tang, Shixiang Liu, Jiaheng Ling, Fenghua Lu, Yan Ren, Yuchen Fei, Ben Zhao, Zhen Gu, Xinyu Su, Rui Wu, Xiao-Ming Si, Weikang Liu, Yang Chen, Hao Yan, Xiangchao Yang, Xue Yan, Junchi Wu, Jiamin Zheng, Qihao Li, Chenhui Gao, Zhiqiang Kong, Hao He, Junjun Su, Mao Fu, Tianfan Ye, Peng Song, Chunfeng Dong, Nanqing Li, Yuqiang Fu, Huazhu Sun, Siqi Cheng, Lijing Lin, Jintai Ouyang, Wanli Zhou, Bowen Zhang, Wenlong Bai, Lei
contents	Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
format	Preprint
id	arxiv_https___arxiv_org_abs_2512_16969
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows Xu, Wanghan Zhou, Yuhao Zhou, Yifan Cao, Qinglong Li, Shuo Bu, Jia Liu, Bo Chen, Yixin He, Xuming Zhao, Xiangyu Zhuang, Xiang Wang, Fengxiang Zhou, Zhiwang Feng, Qiantai Huang, Wenxuan Wei, Jiaqi Wu, Hao Yang, Yuejin Wang, Guangshuai Xu, Sheng Huang, Ziyan Liu, Xinyao Liu, Jiyao Tang, Cheng Li, Wei Chen, Ying Ning, Junzhi Jiang, Pengfei Ma, Chenglong Du, Ye Ji, Changkai Xu, Huihui Hu, Ming Zheng, Jiangbin Chen, Xin Wu, Yucheng Jiang, Feifei Chen, Xi Tang, Xiangru Fu, Yuchen Lu, Yingzhou Zhang, Yuanyuan Sun, Lihao Li, Chengbo Ma, Jinzhe Liu, Wanhao Liu, Yating Wu, Kuo-Cheng Chai, Shengdu Wang, Yizhou Zhangjin, Ouwen Tang, Chen Zhang, Shufei Cao, Wenbo Ren, Junjie Cui, Taoyong Yao, Zhouheng Deng, Juntao Sun, Yijie Liu, Feng Wei, Wangxu Xu, Jingyi Li, Zhangrui Gong, Junchao Guo, Zijie Yao, Zhiyu Chen, Zaoyu Peng, Tianhao Yu, Fangchen Zhang, Bo Zhou, Dongzhan Tang, Shixiang Liu, Jiaheng Ling, Fenghua Lu, Yan Ren, Yuchen Fei, Ben Zhao, Zhen Gu, Xinyu Su, Rui Wu, Xiao-Ming Si, Weikang Liu, Yang Chen, Hao Yan, Xiangchao Yang, Xue Yan, Junchi Wu, Jiamin Zheng, Qihao Li, Chenhui Gao, Zhiqiang Kong, Hao He, Junjun Su, Mao Fu, Tianfan Ye, Peng Song, Chunfeng Dong, Nanqing Li, Yuqiang Fu, Huazhu Sun, Siqi Cheng, Lijing Lin, Jintai Ouyang, Wanli Zhou, Bowen Zhang, Wenlong Bai, Lei Artificial Intelligence Computation and Language Machine Learning Despite advances in scientific AI, a coherent framework for Scientific General Intelligence (SGI)-the ability to autonomously conceive, investigate, and reason across scientific domains-remains lacking. We present an operational SGI definition grounded in the Practical Inquiry Model (PIM: Deliberation, Conception, Action, Perception) and operationalize it via four scientist-aligned tasks: deep research, idea generation, dry/wet experiments, and experimental reasoning. SGI-Bench comprises over 1,000 expert-curated, cross-disciplinary samples inspired by Science's 125 Big Questions, enabling systematic evaluation of state-of-the-art LLMs. Results reveal gaps: low exact match (10--20%) in deep research despite step-level alignment; ideas lacking feasibility and detail; high code executability but low execution result accuracy in dry experiments; low sequence fidelity in wet protocols; and persistent multimodal comparative-reasoning challenges. We further introduce Test-Time Reinforcement Learning (TTRL), which optimizes retrieval-augmented novelty rewards at inference, enhancing hypothesis novelty without reference answer. Together, our PIM-grounded definition, workflow-centric benchmark, and empirical insights establish a foundation for AI systems that genuinely participate in scientific discovery.
title	Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
topic	Artificial Intelligence Computation and Language Machine Learning
url	https://arxiv.org/abs/2512.16969