Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.16834 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913829003198464 |
|---|---|
| author | Birrell, Jeremiah |
| author_facet | Birrell, Jeremiah |
| contents | Generative adversarial networks (GANs) are unsupervised learning methods for training a generator distribution to produce samples that approximate those drawn from a target distribution. Many such methods can be formulated as minimization of a metric or divergence between probability distributions. Recent works have derived statistical error bounds for GANs that are based on integral probability metrics (IPMs), e.g., WGAN which is based on the 1-Wasserstein metric. In general, IPMs are defined by optimizing a linear functional (difference of expectations) over a space of discriminators. A much larger class of GANs, which we here call $(f,Γ)$-GANs, can be constructed using $f$-divergences (e.g., Jensen-Shannon, KL, or $α$-divergences) together with a regularizing discriminator space $Γ$ (e.g., $1$-Lipschitz functions). These GANs have nonlinear objective functions, depending on the choice of $f$, and have been shown to exhibit improved performance in a number of applications. In this work we derive statistical error bounds for $(f,Γ)$-GANs for general classes of $f$ and $Γ$ in the form of finite-sample concentration inequalities. These results prove the statistical consistency of $(f,Γ)$-GANs and reduce to the known results for IPM-GANs in the appropriate limit. Our results use novel Rademacher complexity bounds which provide new insight into the performance of IPM-GANs for distributions with unbounded support and have application to statistical learning tasks beyond GANs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_16834 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Statistical Error Bounds for GANs with Nonlinear Objective Functionals Birrell, Jeremiah Machine Learning Generative adversarial networks (GANs) are unsupervised learning methods for training a generator distribution to produce samples that approximate those drawn from a target distribution. Many such methods can be formulated as minimization of a metric or divergence between probability distributions. Recent works have derived statistical error bounds for GANs that are based on integral probability metrics (IPMs), e.g., WGAN which is based on the 1-Wasserstein metric. In general, IPMs are defined by optimizing a linear functional (difference of expectations) over a space of discriminators. A much larger class of GANs, which we here call $(f,Γ)$-GANs, can be constructed using $f$-divergences (e.g., Jensen-Shannon, KL, or $α$-divergences) together with a regularizing discriminator space $Γ$ (e.g., $1$-Lipschitz functions). These GANs have nonlinear objective functions, depending on the choice of $f$, and have been shown to exhibit improved performance in a number of applications. In this work we derive statistical error bounds for $(f,Γ)$-GANs for general classes of $f$ and $Γ$ in the form of finite-sample concentration inequalities. These results prove the statistical consistency of $(f,Γ)$-GANs and reduce to the known results for IPM-GANs in the appropriate limit. Our results use novel Rademacher complexity bounds which provide new insight into the performance of IPM-GANs for distributions with unbounded support and have application to statistical learning tasks beyond GANs. |
| title | Statistical Error Bounds for GANs with Nonlinear Objective Functionals |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2406.16834 |