Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908593174872064 |
|---|---|
| author | Bhattacharya, Antara Raaghavi Papadimitriou, Isabel Davidson, Kathryn Alvarez-Melis, David |
| author_facet | Bhattacharya, Antara Raaghavi Papadimitriou, Isabel Davidson, Kathryn Alvarez-Melis, David |
| contents | Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguistic-mathematical puzzles involving cross-linguistic numeral systems, which humans can learn to solve successfully. We investigate why this task is difficult for LLMs through a series of experiments that untangle the linguistic and mathematical aspects of numbers in language. Our experiments establish that models cannot consistently solve such problems unless the mathematical operations in the problems are explicitly marked using known symbols ($+$, $\times$, etc., as in "twenty + three"). In further ablation studies, we probe how individual parameters of numeral construction and combination affect performance. While humans use their linguistic understanding of numbers to make inferences about the implicit compositional structure of numerals, LLMs seem to lack this notion of implicit numeral structure. We conclude that the ability to flexibly infer compositional rules from implicit patterns in human-scale data remains an open challenge for current reasoning models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_13886 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles Bhattacharya, Antara Raaghavi Papadimitriou, Isabel Davidson, Kathryn Alvarez-Melis, David Computation and Language Artificial Intelligence Across languages, numeral systems vary widely in how they construct and combine numbers. While humans consistently learn to navigate this diversity, large language models (LLMs) struggle with linguistic-mathematical puzzles involving cross-linguistic numeral systems, which humans can learn to solve successfully. We investigate why this task is difficult for LLMs through a series of experiments that untangle the linguistic and mathematical aspects of numbers in language. Our experiments establish that models cannot consistently solve such problems unless the mathematical operations in the problems are explicitly marked using known symbols ($+$, $\times$, etc., as in "twenty + three"). In further ablation studies, we probe how individual parameters of numeral construction and combination affect performance. While humans use their linguistic understanding of numbers to make inferences about the implicit compositional structure of numerals, LLMs seem to lack this notion of implicit numeral structure. We conclude that the ability to flexibly infer compositional rules from implicit patterns in human-scale data remains an open challenge for current reasoning models. |
| title | Investigating the interaction of linguistic and mathematical reasoning in language models using multilingual number puzzles |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2506.13886 |