Optimization Benchmark for Diffusion Models on Dynamical Systems
Fuente:
arXiv
Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911226636795904 |
|---|---|
| author | Schaipp, Fabian |
| author_facet | Schaipp, Fabian |
| contents | The training of diffusion models is often absent in the evaluation of new optimization techniques. In this work, we benchmark recent optimization algorithms for training a diffusion model for denoising flow trajectories. We observe that Muon and SOAP are highly efficient alternatives to AdamW (18% lower final loss). We also revisit several recent phenomena related to the training of models for text or image applications in the context of diffusion model training. This includes the impact of the learning-rate schedule on the training dynamics, and the performance gap between Adam and SGD. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19376 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimization Benchmark for Diffusion Models on Dynamical Systems Schaipp, Fabian Machine Learning Optimization and Control The training of diffusion models is often absent in the evaluation of new optimization techniques. In this work, we benchmark recent optimization algorithms for training a diffusion model for denoising flow trajectories. We observe that Muon and SOAP are highly efficient alternatives to AdamW (18% lower final loss). We also revisit several recent phenomena related to the training of models for text or image applications in the context of diffusion model training. This includes the impact of the learning-rate schedule on the training dynamics, and the performance gap between Adam and SGD. |
| title | Optimization Benchmark for Diffusion Models on Dynamical Systems |
| topic | Machine Learning Optimization and Control |
| url | https://arxiv.org/abs/2510.19376 |