Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912836448419840 |
|---|---|
| author | Dixit, Aradhya Liang, Tianxi Telang, Jai |
| author_facet | Dixit, Aradhya Liang, Tianxi Telang, Jai |
| contents | Small Language Models (SLMs, under 10B parameters) are attractive for private, on-device deployment, yet they frequently fail on strict constraint-satisfaction problems due to linear, overconfident reasoning traces that do not recover from early mistakes. We introduce Verifier-Guided Distillation, a training protocol that transfers the process of error repair - explicit conflict detection and backtracking - rather than only correct final answers. By training a 7B model on verified reasoning traces that include mistakes and self-corrections, we show that latent verification behavior can emerge in small models, enabling them to occasionally stop, detect contradictions, and revise earlier assumptions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_14290 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models Dixit, Aradhya Liang, Tianxi Telang, Jai Computation and Language Small Language Models (SLMs, under 10B parameters) are attractive for private, on-device deployment, yet they frequently fail on strict constraint-satisfaction problems due to linear, overconfident reasoning traces that do not recover from early mistakes. We introduce Verifier-Guided Distillation, a training protocol that transfers the process of error repair - explicit conflict detection and backtracking - rather than only correct final answers. By training a 7B model on verified reasoning traces that include mistakes and self-corrections, we show that latent verification behavior can emerge in small models, enabling them to occasionally stop, detect contradictions, and revise earlier assumptions. |
| title | Project Aletheia: Verifier-Guided Distillation of Backtracking for Small Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.14290 |