Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866918525902258176 |
|---|---|
| author | Lee, Jaewoo Kang, Hyeongyu Kim, Dohyun Sim, Kyuil Shin, Woocheol Kim, Minsu Yun, Taeyoung Lee, Jeongjae Choi, Sanghyeok Lee, Tabitha Edith Ye, Jong Chul Park, Jinkyoo |
| author_facet | Lee, Jaewoo Kang, Hyeongyu Kim, Dohyun Sim, Kyuil Shin, Woocheol Kim, Minsu Yun, Taeyoung Lee, Jeongjae Choi, Sanghyeok Lee, Tabitha Edith Ye, Jong Chul Park, Jinkyoo |
| contents | Aligning a few-step generative model is challenging, since existing alignment frameworks typically rely on restrictive assumptions: a tractable likelihood, a specific ODE/SDE solver, or a particular model family. We introduce FAV, Few-step Generative Models Alignment via Sample-based Variational Inference, a general alignment framework that requires only sample access to the generator and the reference distribution. We cast alignment as sampling from a reward-tilted distribution anchored to a reference distribution. We leverage Stein Variational Gradient Descent as a sample-based variational inference scheme and amortize its particle updates into the generator parameters via fixed-point regression. We evaluate FAV on two domains: robotics manipulation and image generator alignment. On generative policy alignment for robotic manipulation, FAV outperforms prevailing policy extraction baselines across 56 offline and 30 offline-to-online RL tasks. For image generator alignment, FAV fine-tunes diverse few-step backbones, including GAN, drifting model, consistency models, and flow maps, scaling from ImageNet-$256$ to 1024$^2$ text-to-image synthesis. Code is available at https://github.com/Jaewoopudding/FAV. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_26552 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference Lee, Jaewoo Kang, Hyeongyu Kim, Dohyun Sim, Kyuil Shin, Woocheol Kim, Minsu Yun, Taeyoung Lee, Jeongjae Choi, Sanghyeok Lee, Tabitha Edith Ye, Jong Chul Park, Jinkyoo Machine Learning Artificial Intelligence Aligning a few-step generative model is challenging, since existing alignment frameworks typically rely on restrictive assumptions: a tractable likelihood, a specific ODE/SDE solver, or a particular model family. We introduce FAV, Few-step Generative Models Alignment via Sample-based Variational Inference, a general alignment framework that requires only sample access to the generator and the reference distribution. We cast alignment as sampling from a reward-tilted distribution anchored to a reference distribution. We leverage Stein Variational Gradient Descent as a sample-based variational inference scheme and amortize its particle updates into the generator parameters via fixed-point regression. We evaluate FAV on two domains: robotics manipulation and image generator alignment. On generative policy alignment for robotic manipulation, FAV outperforms prevailing policy extraction baselines across 56 offline and 30 offline-to-online RL tasks. For image generator alignment, FAV fine-tunes diverse few-step backbones, including GAN, drifting model, consistency models, and flow maps, scaling from ImageNet-$256$ to 1024$^2$ text-to-image synthesis. Code is available at https://github.com/Jaewoopudding/FAV. |
| title | Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2605.26552 |