Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2502.12821 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912407752802304 |
|---|---|
| author | Stringli, Elena Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos |
| author_facet | Stringli, Elena Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos |
| contents | Inverse tasks can uncover potential reasoning gaps as Large Language Models (LLMs) scale up. In this work, we explore the redefinition task, in which we assign alternative values to well-known physical constants and units of measure, prompting LLMs to respond accordingly. Our findings show that not only does model performance degrade with scale, but its false confidence also rises. Moreover, while factors such as prompting strategies or response formatting are influential, they do not preclude LLMs from anchoring to memorized values. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12821 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models Stringli, Elena Lymperaiou, Maria Filandrianos, Giorgos Voulodimos, Athanasios Stamou, Giorgos Computation and Language Inverse tasks can uncover potential reasoning gaps as Large Language Models (LLMs) scale up. In this work, we explore the redefinition task, in which we assign alternative values to well-known physical constants and units of measure, prompting LLMs to respond accordingly. Our findings show that not only does model performance degrade with scale, but its false confidence also rises. Moreover, while factors such as prompting strategies or response formatting are influential, they do not preclude LLMs from anchoring to memorized values. |
| title | Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2502.12821 |