SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909987367813120 |
|---|---|
| author | Su, Encheng Wu, Jianyu Tang, Chen Wang, Lintao Li, Pengze Wang, Aoran Zhang, Jinouwen Wang, Yizhou Meng, Yuan Ma, Xinzhu Tang, Shixiang Li, Houqiang |
| author_facet | Su, Encheng Wu, Jianyu Tang, Chen Wang, Lintao Li, Pengze Wang, Aoran Zhang, Jinouwen Wang, Yizhou Meng, Yuan Ma, Xinzhu Tang, Shixiang Li, Houqiang |
| contents | As large language models (LLMs) transition from general knowledge retrieval to complex scientific discovery, their evaluation standards must also incorporate the rigorous norms of scientific inquiry. Existing benchmarks exhibit a critical blind spot: general instruction-following metrics focus on superficial formatting, while domain-specific scientific benchmarks assess only final-answer correctness, often rewarding models that arrive at the right result with the wrong reasons. To address this gap, we introduce scientific instruction following: the capability to solve problems while strictly adhering to the constraints that establish scientific validity. Specifically, we introduce SciIF, a multi-discipline benchmark that evaluates this capability by pairing university-level problems with a fixed catalog of constraints across three pillars: scientific conditions (e.g., boundary checks and assumptions), semantic stability (e.g., unit and symbol conventions), and specific processes(e.g., required numerical methods). Uniquely, SciIF emphasizes auditability, requiring models to provide explicit evidence of constraint satisfaction rather than implicit compliance. By measuring both solution correctness and multi-constraint adherence, SciIF enables finegrained diagnosis of compositional reasoning failures, ensuring that LLMs can function as reliable agents within the strict logical frameworks of science. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04770 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence Su, Encheng Wu, Jianyu Tang, Chen Wang, Lintao Li, Pengze Wang, Aoran Zhang, Jinouwen Wang, Yizhou Meng, Yuan Ma, Xinzhu Tang, Shixiang Li, Houqiang Artificial Intelligence Databases As large language models (LLMs) transition from general knowledge retrieval to complex scientific discovery, their evaluation standards must also incorporate the rigorous norms of scientific inquiry. Existing benchmarks exhibit a critical blind spot: general instruction-following metrics focus on superficial formatting, while domain-specific scientific benchmarks assess only final-answer correctness, often rewarding models that arrive at the right result with the wrong reasons. To address this gap, we introduce scientific instruction following: the capability to solve problems while strictly adhering to the constraints that establish scientific validity. Specifically, we introduce SciIF, a multi-discipline benchmark that evaluates this capability by pairing university-level problems with a fixed catalog of constraints across three pillars: scientific conditions (e.g., boundary checks and assumptions), semantic stability (e.g., unit and symbol conventions), and specific processes(e.g., required numerical methods). Uniquely, SciIF emphasizes auditability, requiring models to provide explicit evidence of constraint satisfaction rather than implicit compliance. By measuring both solution correctness and multi-constraint adherence, SciIF enables finegrained diagnosis of compositional reasoning failures, ensuring that LLMs can function as reliable agents within the strict logical frameworks of science. |
| title | SciIF: Benchmarking Scientific Instruction Following Towards Rigorous Scientific Intelligence |
| topic | Artificial Intelligence Databases |
| url | https://arxiv.org/abs/2601.04770 |