CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Prediction
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911138932850688 |
|---|---|
| author | Chen, Yuzhu Wang, Yingjie Liu, Shunyu Jing, Yongcheng Tao, Dacheng |
| author_facet | Chen, Yuzhu Wang, Yingjie Liu, Shunyu Jing, Yongcheng Tao, Dacheng |
| contents | Autoregressive pre-trained models combined with decoding methods have achieved impressive performance on complex reasoning tasks. While mainstream decoding strategies such as beam search can generate plausible candidate sets, they often lack provable coverage guarantees, and struggle to effectively balance search efficiency with the need for versatile trajectories, particularly those involving long-tail sequences that are essential in certain real-world applications. To address these limitations, we propose \textsc{CoVeR}, a novel model-free decoding strategy wihtin the conformal prediction framework that simultaneously maintains a compact search space and ensures high coverage probability over desirable trajectories. Theoretically, we establish a PAC-style generalization bound, guaranteeing that \textsc{CoVeR} asymptotically achieves a coverage rate of at least $1 - α$ for any target level $α\in (0,1)$. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_04733 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Prediction Chen, Yuzhu Wang, Yingjie Liu, Shunyu Jing, Yongcheng Tao, Dacheng Machine Learning Artificial Intelligence Autoregressive pre-trained models combined with decoding methods have achieved impressive performance on complex reasoning tasks. While mainstream decoding strategies such as beam search can generate plausible candidate sets, they often lack provable coverage guarantees, and struggle to effectively balance search efficiency with the need for versatile trajectories, particularly those involving long-tail sequences that are essential in certain real-world applications. To address these limitations, we propose \textsc{CoVeR}, a novel model-free decoding strategy wihtin the conformal prediction framework that simultaneously maintains a compact search space and ensures high coverage probability over desirable trajectories. Theoretically, we establish a PAC-style generalization bound, guaranteeing that \textsc{CoVeR} asymptotically achieves a coverage rate of at least $1 - α$ for any target level $α\in (0,1)$. |
| title | CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Prediction |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2509.04733 |