Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2509.23529 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911262943739904 |
|---|---|
| author | Kuleshov, Ilya Pavel, Ilin Kompanets, Nikolay Sycheva, Ksenia Nikolich, Aleksandr |
| author_facet | Kuleshov, Ilya Pavel, Ilin Kompanets, Nikolay Sycheva, Ksenia Nikolich, Aleksandr |
| contents | This paper introduces DOoM, a new open-source benchmark designed to assess the capabilities of language models in solving mathematics and physics problems in Russian. The benchmark includes problems of varying difficulty, ranging from school-level tasks to university Olympiad and entrance exam questions. In this paper we discuss the motivation behind its creation, describe dataset's structure and evaluation methodology, and present initial results from testing various models. Analysis of the results shows a correlation between model performance and the number of tokens used, and highlights differences in performance between mathematics and physics tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_23529 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DOoM: Difficult Olympiads of Math Kuleshov, Ilya Pavel, Ilin Kompanets, Nikolay Sycheva, Ksenia Nikolich, Aleksandr Artificial Intelligence This paper introduces DOoM, a new open-source benchmark designed to assess the capabilities of language models in solving mathematics and physics problems in Russian. The benchmark includes problems of varying difficulty, ranging from school-level tasks to university Olympiad and entrance exam questions. In this paper we discuss the motivation behind its creation, describe dataset's structure and evaluation methodology, and present initial results from testing various models. Analysis of the results shows a correlation between model performance and the number of tokens used, and highlights differences in performance between mathematics and physics tasks. |
| title | DOoM: Difficult Olympiads of Math |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2509.23529 |