Is RL fine-tuning harder than regression? A PDE learning approach for diffusion models
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911134651514880 |
|---|---|
| author | Mou, Wenlong |
| author_facet | Mou, Wenlong |
| contents | We study the problem of learning the optimal control policy for fine-tuning a given diffusion process, using general value function approximation. We develop a new class of algorithms by solving a variational inequality problem based on the Hamilton-Jacobi-Bellman (HJB) equations. We prove sharp statistical rates for the learned value function and control policy, depending on the complexity and approximation errors of the function class. In contrast to generic reinforcement learning problems, our approach shows that fine-tuning can be achieved via supervised regression, with faster statistical rate guarantees. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_02528 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Is RL fine-tuning harder than regression? A PDE learning approach for diffusion models Mou, Wenlong Machine Learning Optimization and Control Probability Statistics Theory We study the problem of learning the optimal control policy for fine-tuning a given diffusion process, using general value function approximation. We develop a new class of algorithms by solving a variational inequality problem based on the Hamilton-Jacobi-Bellman (HJB) equations. We prove sharp statistical rates for the learned value function and control policy, depending on the complexity and approximation errors of the function class. In contrast to generic reinforcement learning problems, our approach shows that fine-tuning can be achieved via supervised regression, with faster statistical rate guarantees. |
| title | Is RL fine-tuning harder than regression? A PDE learning approach for diffusion models |
| topic | Machine Learning Optimization and Control Probability Statistics Theory |
| url | https://arxiv.org/abs/2509.02528 |