Learnable Chernoff Baselines for Inference-Time Alignment
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914328271126528 |
|---|---|
| author | Madhow, Sunil Liang, Yuchen Shroff, Ness Liang, Yingbin Wang, Yu-Xiang |
| author_facet | Madhow, Sunil Liang, Yuchen Shroff, Ness Liang, Yingbin Wang, Yu-Xiang |
| contents | We study inference-time reward-guided alignment for generative models. Existing methods often rely on either architecture-specific adaptations or computationally costly inference procedures. We introduce Learnable Chernoff Baselines (LCBs) as a method for efficiently and approximately sampling from the exponentially tilted kernels that arise from KL-regularized reward alignment. Using only black-box sampling access to the pretrained model, LCBs implement a form of rejection sampling with adaptively selected acceptance probabilities, which allows fine-grained control over inference-compute scaling. We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling while using substantially fewer queries to the pretrained model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_07738 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Learnable Chernoff Baselines for Inference-Time Alignment Madhow, Sunil Liang, Yuchen Shroff, Ness Liang, Yingbin Wang, Yu-Xiang Machine Learning Artificial Intelligence We study inference-time reward-guided alignment for generative models. Existing methods often rely on either architecture-specific adaptations or computationally costly inference procedures. We introduce Learnable Chernoff Baselines (LCBs) as a method for efficiently and approximately sampling from the exponentially tilted kernels that arise from KL-regularized reward alignment. Using only black-box sampling access to the pretrained model, LCBs implement a form of rejection sampling with adaptively selected acceptance probabilities, which allows fine-grained control over inference-compute scaling. We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling while using substantially fewer queries to the pretrained model. |
| title | Learnable Chernoff Baselines for Inference-Time Alignment |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2602.07738 |