Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866913028202561536 |
|---|---|
| author | Zheng, Shunfeng Zhang, Yudi Fang, Meng Zhang, Zihan Wu, Zhitan Pechenizkiy, Mykola Chen, Ling |
| author_facet | Zheng, Shunfeng Zhang, Yudi Fang, Meng Zhang, Zihan Wu, Zhitan Pechenizkiy, Mykola Chen, Ling |
| contents | Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for expert-level reasoning-such as solving Olympiad-level physics problems-remains largely unexplored. Inspired by the way students prepare for competitions by reviewing past problems, we investigate the potential of RAG to enhance physics reasoning in foundation models. We introduce PhoPile, a high-quality multimodal dataset specifically designed for Olympiad-level physics, enabling systematic study of retrieval-based reasoning. PhoPile includes diagrams, graphs, and equations, capturing the inherently multimodal nature of physics problem solving. Using PhoPile, we benchmark RAG-augmented foundation models, covering both large language models (LLMs) and large multimodal models (LMMs) with multiple retrievers. Our results demonstrate that integrating retrieval with physics corpora can improve model performance, while also highlighting challenges that motivate further research in retrieval-augmented physics reasoning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_00919 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving Zheng, Shunfeng Zhang, Yudi Fang, Meng Zhang, Zihan Wu, Zhitan Pechenizkiy, Mykola Chen, Ling Computation and Language Artificial Intelligence Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for expert-level reasoning-such as solving Olympiad-level physics problems-remains largely unexplored. Inspired by the way students prepare for competitions by reviewing past problems, we investigate the potential of RAG to enhance physics reasoning in foundation models. We introduce PhoPile, a high-quality multimodal dataset specifically designed for Olympiad-level physics, enabling systematic study of retrieval-based reasoning. PhoPile includes diagrams, graphs, and equations, capturing the inherently multimodal nature of physics problem solving. Using PhoPile, we benchmark RAG-augmented foundation models, covering both large language models (LLMs) and large multimodal models (LMMs) with multiple retrievers. Our results demonstrate that integrating retrieval with physics corpora can improve model performance, while also highlighting challenges that motivate further research in retrieval-augmented physics reasoning. |
| title | Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.00919 |