MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866918472768815104 |
|---|---|
| author | Teixeira, Tiago Erthal, Ana Carolina Belieni, Juan Canaverde, Beatriz Mesquita, Diego Faria, Miguel da Silva, Eliezer de Souza Martins, André F. T. |
| author_facet | Teixeira, Tiago Erthal, Ana Carolina Belieni, Juan Canaverde, Beatriz Mesquita, Diego Faria, Miguel da Silva, Eliezer de Souza Martins, André F. T. |
| contents | The use of large language models (LLMs) for complex mathematical reasoning is an emergent area of research, with fast progress in methods, models, and benchmark datasets. However, most mathematical reasoning evaluations exhibit a significant linguistic bias, with the vast majority of benchmark datasets being exclusively in English or (at best) translated from English. We address this limitation by introducing {\sc Math-PT}, a novel dataset comprising 1,729 mathematical problems written in European and Brazilian Portuguese. {\sc Math-PT} is curated from a variety of high-quality native sources, including mathematical Olympiads, competitions, and exams from Portugal and Brazil. We present a comprehensive benchmark of current state-of-the-art LLMs on {\sc Math-PT}, revealing that frontier reasoning models achieve strong performance in multiple choice questions compared to open weight models, but that their performance decreases for questions with figures or open-ended questions. To facilitate future research, we release the benchmark dataset and model outputs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_25926 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese Teixeira, Tiago Erthal, Ana Carolina Belieni, Juan Canaverde, Beatriz Mesquita, Diego Faria, Miguel da Silva, Eliezer de Souza Martins, André F. T. Computation and Language Information Retrieval I.2.7; I.2.0 The use of large language models (LLMs) for complex mathematical reasoning is an emergent area of research, with fast progress in methods, models, and benchmark datasets. However, most mathematical reasoning evaluations exhibit a significant linguistic bias, with the vast majority of benchmark datasets being exclusively in English or (at best) translated from English. We address this limitation by introducing {\sc Math-PT}, a novel dataset comprising 1,729 mathematical problems written in European and Brazilian Portuguese. {\sc Math-PT} is curated from a variety of high-quality native sources, including mathematical Olympiads, competitions, and exams from Portugal and Brazil. We present a comprehensive benchmark of current state-of-the-art LLMs on {\sc Math-PT}, revealing that frontier reasoning models achieve strong performance in multiple choice questions compared to open weight models, but that their performance decreases for questions with figures or open-ended questions. To facilitate future research, we release the benchmark dataset and model outputs. |
| title | MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese |
| topic | Computation and Language Information Retrieval I.2.7; I.2.0 |
| url | https://arxiv.org/abs/2604.25926 |