Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2603.01589 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913000339800064 |
|---|---|
| author | Zhu, Xiangyang Tian, Yuan Jia, Qi Zhang, Kaiwei Zhang, Zicheng Li, Chunyi Ji, Kaiyuan Liu, Dongrui Chen, Zijian Sun, Lu Zhang, Renrui Teng, Yan Shao, Jing Sun, Wei Hu, Xia Qiao, Yu Zhai, Guangtao |
| author_facet | Zhu, Xiangyang Tian, Yuan Jia, Qi Zhang, Kaiwei Zhang, Zicheng Li, Chunyi Ji, Kaiyuan Liu, Dongrui Chen, Zijian Sun, Lu Zhang, Renrui Teng, Yan Shao, Jing Sun, Wei Hu, Xia Qiao, Yu Zhai, Guangtao |
| contents | The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs, revealing critical vulnerabilities in current models. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of models. Finally, we argue that knowledge is a double-edged sword, and determining the safety of a scientific question should depend on specific context, rather than universally categorizing it as safe or unsafe. Our work provides both a diagnostic tool and a practical resource for building safer scientific AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01589 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond Zhu, Xiangyang Tian, Yuan Jia, Qi Zhang, Kaiwei Zhang, Zicheng Li, Chunyi Ji, Kaiyuan Liu, Dongrui Chen, Zijian Sun, Lu Zhang, Renrui Teng, Yan Shao, Jing Sun, Wei Hu, Xia Qiao, Yu Zhai, Guangtao Machine Learning Artificial Intelligence The success of large language models (LLMs) in scientific domains has heightened safety concerns, prompting numerous benchmarks to evaluate their scientific safety. Existing benchmarks often suffer from limited risk coverage and a reliance on subjective evaluation. To address these problems, we introduce SafeSci, a comprehensive framework for safety evaluation and enhancement in scientific contexts. SafeSci comprises SafeSciBench, a multi-disciplinary benchmark with 0.25M samples, and SafeSciTrain, a large-scale dataset containing 1.5M samples for safety enhancement. SafeSciBench distinguishes between safety knowledge and risk to cover extensive scopes and employs objective metrics such as deterministically answerable questions to mitigate evaluation bias. We evaluate 24 advanced LLMs, revealing critical vulnerabilities in current models. We also observe that LLMs exhibit varying degrees of excessive refusal behaviors on safety-related issues. For safety enhancement, we demonstrate that fine-tuning on SafeSciTrain significantly enhances the safety alignment of models. Finally, we argue that knowledge is a double-edged sword, and determining the safety of a scientific question should depend on specific context, rather than universally categorizing it as safe or unsafe. Our work provides both a diagnostic tool and a practical resource for building safer scientific AI systems. |
| title | SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2603.01589 |