Minimizing $f$-Divergences by Interpolating Velocity Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909217779089408 |
|---|---|
| author | Liu, Song Yu, Jiahao Simons, Jack Yi, Mingxuan Beaumont, Mark |
| author_facet | Liu, Song Yu, Jiahao Simons, Jack Yi, Mingxuan Beaumont, Mark |
| contents | Many machine learning problems can be seen as approximating a \textit{target} distribution using a \textit{particle} distribution by minimizing their statistical discrepancy. Wasserstein Gradient Flow can move particles along a path that minimizes the $f$-divergence between the target and particle distributions. To move particles, we need to calculate the corresponding velocity fields derived from a density ratio function between these two distributions. Previous works estimated such density ratio functions and then differentiated the estimated ratios. These approaches may suffer from overfitting, leading to a less accurate estimate of the velocity fields. Inspired by non-parametric curve fitting, we directly estimate these velocity fields using interpolation techniques. We prove that our estimators are consistent under mild conditions. We validate their effectiveness using novel applications on domain adaptation and missing data imputation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_15577 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Minimizing $f$-Divergences by Interpolating Velocity Fields Liu, Song Yu, Jiahao Simons, Jack Yi, Mingxuan Beaumont, Mark Machine Learning Many machine learning problems can be seen as approximating a \textit{target} distribution using a \textit{particle} distribution by minimizing their statistical discrepancy. Wasserstein Gradient Flow can move particles along a path that minimizes the $f$-divergence between the target and particle distributions. To move particles, we need to calculate the corresponding velocity fields derived from a density ratio function between these two distributions. Previous works estimated such density ratio functions and then differentiated the estimated ratios. These approaches may suffer from overfitting, leading to a less accurate estimate of the velocity fields. Inspired by non-parametric curve fitting, we directly estimate these velocity fields using interpolation techniques. We prove that our estimators are consistent under mild conditions. We validate their effectiveness using novel applications on domain adaptation and missing data imputation. |
| title | Minimizing $f$-Divergences by Interpolating Velocity Fields |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2305.15577 |