CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916450449489920 |
|---|---|
| author | Lee, Chia-Hsuan Cheng, Hao Ostendorf, Mari |
| author_facet | Lee, Chia-Hsuan Cheng, Hao Ostendorf, Mari |
| contents | Large language models (LLMs) have demonstrated self-improvement capabilities via feedback and refinement, but current small language models (SLMs) have had limited success in this area. Existing correction approaches often rely on distilling knowledge from LLMs, which imposes significant computation demands. In this work, we introduce CORRECTIONLM, a novel correction framework that enables SLMs to self-correct using in-context exemplars without LLM involvement. Applied to two dialogue state tracking (DST) tasks in low-resource settings, CORRECTIONLM achieves results similar to a state-of-the-art LLM at a small fraction of the computation costs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_18209 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking Lee, Chia-Hsuan Cheng, Hao Ostendorf, Mari Computation and Language Large language models (LLMs) have demonstrated self-improvement capabilities via feedback and refinement, but current small language models (SLMs) have had limited success in this area. Existing correction approaches often rely on distilling knowledge from LLMs, which imposes significant computation demands. In this work, we introduce CORRECTIONLM, a novel correction framework that enables SLMs to self-correct using in-context exemplars without LLM involvement. Applied to two dialogue state tracking (DST) tasks in low-resource settings, CORRECTIONLM achieves results similar to a state-of-the-art LLM at a small fraction of the computation costs. |
| title | CorrectionLM: Self-Corrections with SLM for Dialogue State Tracking |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2410.18209 |