QuantumBench: A Benchmark for Quantum Problem Solving
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909880847171584 |
|---|---|
| author | Minami, Shunya Ishigaki, Tatsuya Hamamura, Ikko Mikuriya, Taku Ma, Youmi Okazaki, Naoaki Takamura, Hiroya Suzuki, Yohichi Kadowaki, Tadashi |
| author_facet | Minami, Shunya Ishigaki, Tatsuya Hamamura, Ikko Mikuriya, Taku Ma, Youmi Okazaki, Naoaki Takamura, Hiroya Suzuki, Yohichi Kadowaki, Tadashi |
| contents | Large language models are now integrated into many scientific workflows, accelerating data analysis, hypothesis generation, and design space exploration. In parallel with this growth, there is a growing need to carefully evaluate whether models accurately capture domain-specific knowledge and notation, since general-purpose benchmarks rarely reflect these requirements. This gap is especially clear in quantum science, which features non-intuitive phenomena and requires advanced mathematics. In this study, we introduce QuantumBench, a benchmark for the quantum domain that systematically examine how well LLMs understand and can be applied to this non-intuitive field. Using publicly available materials, we compiled approximately 800 questions with their answers spanning nine areas related to quantum science and organized them into an eight-option multiple-choice dataset. With this benchmark, we evaluate several existing LLMs and analyze their performance in the quantum domain, including sensitivity to changes in question format. QuantumBench is the first LLM evaluation dataset built for the quantum domain, and it is intended to guide the effective use of LLMs in quantum research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_00092 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | QuantumBench: A Benchmark for Quantum Problem Solving Minami, Shunya Ishigaki, Tatsuya Hamamura, Ikko Mikuriya, Taku Ma, Youmi Okazaki, Naoaki Takamura, Hiroya Suzuki, Yohichi Kadowaki, Tadashi Artificial Intelligence Computation and Language Machine Learning Quantum Physics Large language models are now integrated into many scientific workflows, accelerating data analysis, hypothesis generation, and design space exploration. In parallel with this growth, there is a growing need to carefully evaluate whether models accurately capture domain-specific knowledge and notation, since general-purpose benchmarks rarely reflect these requirements. This gap is especially clear in quantum science, which features non-intuitive phenomena and requires advanced mathematics. In this study, we introduce QuantumBench, a benchmark for the quantum domain that systematically examine how well LLMs understand and can be applied to this non-intuitive field. Using publicly available materials, we compiled approximately 800 questions with their answers spanning nine areas related to quantum science and organized them into an eight-option multiple-choice dataset. With this benchmark, we evaluate several existing LLMs and analyze their performance in the quantum domain, including sensitivity to changes in question format. QuantumBench is the first LLM evaluation dataset built for the quantum domain, and it is intended to guide the effective use of LLMs in quantum research. |
| title | QuantumBench: A Benchmark for Quantum Problem Solving |
| topic | Artificial Intelligence Computation and Language Machine Learning Quantum Physics |
| url | https://arxiv.org/abs/2511.00092 |