Dynamic Search for Inference-Time Alignment in Diffusion Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910981760745472 |
|---|---|
| author | Li, Xiner Uehara, Masatoshi Su, Xingyu Scalia, Gabriele Biancalani, Tommaso Regev, Aviv Levine, Sergey Ji, Shuiwang |
| author_facet | Li, Xiner Uehara, Masatoshi Su, Xingyu Scalia, Gabriele Biancalani, Tommaso Regev, Aviv Levine, Sergey Ji, Shuiwang |
| contents | Diffusion models have shown promising generative capabilities across diverse domains, yet aligning their outputs with desired reward functions remains a challenge, particularly in cases where reward functions are non-differentiable. Some gradient-free guidance methods have been developed, but they often struggle to achieve optimal inference-time alignment. In this work, we newly frame inference-time alignment in diffusion as a search problem and propose Dynamic Search for Diffusion (DSearch), which subsamples from denoising processes and approximates intermediate node rewards. It also dynamically adjusts beam width and tree expansion to efficiently explore high-reward generations. To refine intermediate decisions, DSearch incorporates adaptive scheduling based on noise levels and a lookahead heuristic function. We validate DSearch across multiple domains, including biological sequence design, molecular optimization, and image generation, demonstrating superior reward optimization compared to existing approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_02039 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dynamic Search for Inference-Time Alignment in Diffusion Models Li, Xiner Uehara, Masatoshi Su, Xingyu Scalia, Gabriele Biancalani, Tommaso Regev, Aviv Levine, Sergey Ji, Shuiwang Machine Learning Artificial Intelligence Diffusion models have shown promising generative capabilities across diverse domains, yet aligning their outputs with desired reward functions remains a challenge, particularly in cases where reward functions are non-differentiable. Some gradient-free guidance methods have been developed, but they often struggle to achieve optimal inference-time alignment. In this work, we newly frame inference-time alignment in diffusion as a search problem and propose Dynamic Search for Diffusion (DSearch), which subsamples from denoising processes and approximates intermediate node rewards. It also dynamically adjusts beam width and tree expansion to efficiently explore high-reward generations. To refine intermediate decisions, DSearch incorporates adaptive scheduling based on noise levels and a lookahead heuristic function. We validate DSearch across multiple domains, including biological sequence design, molecular optimization, and image generation, demonstrating superior reward optimization compared to existing approaches. |
| title | Dynamic Search for Inference-Time Alignment in Diffusion Models |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2503.02039 |