Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2605.10790 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917481026682880 |
|---|---|
| author | Yao, Zhipeng Li, Dazhou Zhang, Zitong Mahee, Durude Zhu, Fan Zhang, Wenbin He, Xinwei Jin, Yeying Yu, Rui |
| author_facet | Yao, Zhipeng Li, Dazhou Zhang, Zitong Mahee, Durude Zhu, Fan Zhang, Wenbin He, Xinwei Jin, Yeying Yu, Rui |
| contents | Diffusion models have achieved remarkable success, yet their training remains inefficient due to a severe optimization bottleneck, which we term Representation Degradation. As noise levels increase, the outputs of the trained model exhibit progressive structural distortion, which can destabilize training and impair generation quality. Our analysis suggests that this instability is driven by mismatched target recoverability, which is associated with Neural Tangent Kernel (NTK) spectral weakening and effective low-rank behavior. To address this, we propose Elucidated Representation Diffusion (ERD), a plug-and-play framework that dynamically reallocates optimization effort according to effective recoverability. By stabilizing representation learning without external supervision, ERD accelerates convergence and achieves strong empirical performance across diffusion backbones. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_10790 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Elucidating Representation Degradation Problem in Diffusion Model Training Yao, Zhipeng Li, Dazhou Zhang, Zitong Mahee, Durude Zhu, Fan Zhang, Wenbin He, Xinwei Jin, Yeying Yu, Rui Machine Learning Diffusion models have achieved remarkable success, yet their training remains inefficient due to a severe optimization bottleneck, which we term Representation Degradation. As noise levels increase, the outputs of the trained model exhibit progressive structural distortion, which can destabilize training and impair generation quality. Our analysis suggests that this instability is driven by mismatched target recoverability, which is associated with Neural Tangent Kernel (NTK) spectral weakening and effective low-rank behavior. To address this, we propose Elucidated Representation Diffusion (ERD), a plug-and-play framework that dynamically reallocates optimization effort according to effective recoverability. By stabilizing representation learning without external supervision, ERD accelerates convergence and achieves strong empirical performance across diffusion backbones. |
| title | Elucidating Representation Degradation Problem in Diffusion Model Training |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.10790 |