Improving Discriminator Guidance in Diffusion Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866916789742469120 |
|---|---|
| author | Verine, Alexandre Inane, Ahmed Mehdi Bronnec, Florian Le Negrevergne, Benjamin Chevaleyre, Yann |
| author_facet | Verine, Alexandre Inane, Ahmed Mehdi Bronnec, Florian Le Negrevergne, Benjamin Chevaleyre, Yann |
| contents | Discriminator Guidance has become a popular method for efficiently refining pre-trained Score-Matching Diffusion models. However, in this paper, we demonstrate that the standard implementation of this technique does not necessarily lead to a distribution closer to the real data distribution. Specifically, we show that training the discriminator using Cross-Entropy loss, as commonly done, can in fact increase the Kullback-Leibler divergence between the model and target distributions, particularly when the discriminator overfits. To address this, we propose a theoretically sound training objective for discriminator guidance that properly minimizes the KL divergence. We analyze its properties and demonstrate empirically across multiple datasets that our proposed method consistently improves over the conventional method by producing samples of higher quality. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_16117 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Improving Discriminator Guidance in Diffusion Models Verine, Alexandre Inane, Ahmed Mehdi Bronnec, Florian Le Negrevergne, Benjamin Chevaleyre, Yann Machine Learning Discriminator Guidance has become a popular method for efficiently refining pre-trained Score-Matching Diffusion models. However, in this paper, we demonstrate that the standard implementation of this technique does not necessarily lead to a distribution closer to the real data distribution. Specifically, we show that training the discriminator using Cross-Entropy loss, as commonly done, can in fact increase the Kullback-Leibler divergence between the model and target distributions, particularly when the discriminator overfits. To address this, we propose a theoretically sound training objective for discriminator guidance that properly minimizes the KL divergence. We analyze its properties and demonstrate empirically across multiple datasets that our proposed method consistently improves over the conventional method by producing samples of higher quality. |
| title | Improving Discriminator Guidance in Diffusion Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2503.16117 |