The Karp Dataset
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913664807731200 |
|---|---|
| author | DiCicco, Mason Worden, Eamon Olsen, Conner Gangaram, Nikhil Reichman, Daniel Heffernan, Neil |
| author_facet | DiCicco, Mason Worden, Eamon Olsen, Conner Gangaram, Nikhil Reichman, Daniel Heffernan, Neil |
| contents | Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training and benchmarking the performance of LLMs. To this end, we introduce the Karp dataset: The first dataset composed of detailed proofs of NP-completeness reductions. The reductions vary in difficulty, ranging from simple exercises of undergraduate courses to more challenging reductions from academic papers. We compare the performance of state-of-the-art models on this task and demonstrate the effect of fine-tuning with the Karp dataset on reasoning capacity. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_14705 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Karp Dataset DiCicco, Mason Worden, Eamon Olsen, Conner Gangaram, Nikhil Reichman, Daniel Heffernan, Neil Machine Learning Computation and Language Understanding the mathematical reasoning capabilities of Large Language Models (LLMs) is a central topic in the study of artificial intelligence. This new domain necessitates the creation of datasets of reasoning tasks for both training and benchmarking the performance of LLMs. To this end, we introduce the Karp dataset: The first dataset composed of detailed proofs of NP-completeness reductions. The reductions vary in difficulty, ranging from simple exercises of undergraduate courses to more challenging reductions from academic papers. We compare the performance of state-of-the-art models on this task and demonstrate the effect of fine-tuning with the Karp dataset on reasoning capacity. |
| title | The Karp Dataset |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2501.14705 |