Improving Constrained Language Generation via Self-Distilled Twisted Sequential Monte Carlo
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914169651986432 |
|---|---|
| author | Kim, Sooyeon Nam, Giung Park, Byoungwoo Lee, Juho |
| author_facet | Kim, Sooyeon Nam, Giung Park, Byoungwoo Lee, Juho |
| contents | Recent work has framed constrained text generation with autoregressive language models as a probabilistic inference problem. Among these, Zhao et al. (2024) introduced a promising approach based on twisted Sequential Monte Carlo, which incorporates learned twist functions and twist-induced proposals to guide the generation process. However, in constrained generation settings where the target distribution concentrates on outputs that are unlikely under the base model, learning becomes challenging due to sparse and uninformative reward signals. We show that iteratively refining the base model through self-distillation alleviates this issue by making the model progressively more aligned with the target, leading to substantial gains in generation quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_02315 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Constrained Language Generation via Self-Distilled Twisted Sequential Monte Carlo Kim, Sooyeon Nam, Giung Park, Byoungwoo Lee, Juho Machine Learning Recent work has framed constrained text generation with autoregressive language models as a probabilistic inference problem. Among these, Zhao et al. (2024) introduced a promising approach based on twisted Sequential Monte Carlo, which incorporates learned twist functions and twist-induced proposals to guide the generation process. However, in constrained generation settings where the target distribution concentrates on outputs that are unlikely under the base model, learning becomes challenging due to sparse and uninformative reward signals. We show that iteratively refining the base model through self-distillation alleviates this issue by making the model progressively more aligned with the target, leading to substantial gains in generation quality. |
| title | Improving Constrained Language Generation via Self-Distilled Twisted Sequential Monte Carlo |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2507.02315 |