Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913695486967808 |
|---|---|
| author | Huh, Dom Mohapatra, Prasant |
| author_facet | Huh, Dom Mohapatra, Prasant |
| contents | Diffusion-based planning, learning, and control methods present a promising branch of powerful and expressive decision-making solutions. Given the growing interest, such methods have undergone numerous refinements over the past years. However, despite these advancements, existing methods are limited in their investigations regarding general methods for reward maximization within the decision-making process. In this work, we study extensions of fine-tuning approaches for control applications. Specifically, we explore extensions and various design choices for four fine-tuning approaches: reward alignment through reinforcement learning, direct preference optimization, supervised fine-tuning, and cascading diffusion. We optimize their usage to merge these independent efforts into one unified paradigm. We show the utility of such propositions in offline RL settings and demonstrate empirical improvements over a rich array of control tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12198 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control Huh, Dom Mohapatra, Prasant Machine Learning Artificial Intelligence Diffusion-based planning, learning, and control methods present a promising branch of powerful and expressive decision-making solutions. Given the growing interest, such methods have undergone numerous refinements over the past years. However, despite these advancements, existing methods are limited in their investigations regarding general methods for reward maximization within the decision-making process. In this work, we study extensions of fine-tuning approaches for control applications. Specifically, we explore extensions and various design choices for four fine-tuning approaches: reward alignment through reinforcement learning, direct preference optimization, supervised fine-tuning, and cascading diffusion. We optimize their usage to merge these independent efforts into one unified paradigm. We show the utility of such propositions in offline RL settings and demonstrate empirical improvements over a rich array of control tasks. |
| title | Maximize Your Diffusion: A Study into Reward Maximization and Alignment for Diffusion-based Control |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2502.12198 |