Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917338637402112 |
|---|---|
| author | McAllister, David Aittala, Miika Karras, Tero Hellsten, Janne Kanazawa, Angjoo Aila, Timo Laine, Samuli |
| author_facet | McAllister, David Aittala, Miika Karras, Tero Hellsten, Janne Kanazawa, Angjoo Aila, Timo Laine, Samuli |
| contents | Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment. In this paper, we propose an online RL variant that reduces the variance in the model updates by sampling paired trajectories and pulling the flow velocity in the direction of the more favorable image. Unlike existing methods that treat each sampling step as a separate policy action, we consider the entire sampling process as a single action. We experiment with both high-quality vision language models and off-the-shelf quality metrics for rewards, and evaluate the outputs using a broad set of metrics. Our method converges faster and yields higher output quality and prompt alignment than previous approaches. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12893 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models McAllister, David Aittala, Miika Karras, Tero Hellsten, Janne Kanazawa, Angjoo Aila, Timo Laine, Samuli Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Neural and Evolutionary Computing Reinforcement learning (RL) has become a standard technique for post-training diffusion-based image synthesis models, as it enables learning from reward signals to explicitly improve desirable aspects such as image quality and prompt alignment. In this paper, we propose an online RL variant that reduces the variance in the model updates by sampling paired trajectories and pulling the flow velocity in the direction of the more favorable image. Unlike existing methods that treat each sampling step as a separate policy action, we consider the entire sampling process as a single action. We experiment with both high-quality vision language models and off-the-shelf quality metrics for rewards, and evaluate the outputs using a broad set of metrics. Our method converges faster and yields higher output quality and prompt alignment than previous approaches. |
| title | Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Models |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2603.12893 |