TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866915404117442560 |
|---|---|
| author | Bock, Michael R. Molisee, Kara Ozer, Zachary Shah, Sumit |
| author_facet | Bock, Michael R. Molisee, Kara Ozer, Zachary Shah, Sumit |
| contents | Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models' abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_16126 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task Bock, Michael R. Molisee, Kara Ozer, Zachary Shah, Sumit Artificial Intelligence Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models' abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task. |
| title | TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2507.16126 |