Overcoming Saturation in Density Ratio Estimation by Iterated Regularization
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913374950916096 |
|---|---|
| author | Gruber, Lukas Holzleitner, Markus Lehner, Johannes Hochreiter, Sepp Zellinger, Werner |
| author_facet | Gruber, Lukas Holzleitner, Markus Lehner, Johannes Hochreiter, Sepp Zellinger, Werner |
| contents | Estimating the ratio of two probability densities from finitely many samples, is a central task in machine learning and statistics. In this work, we show that a large class of kernel methods for density ratio estimation suffers from error saturation, which prevents algorithms from achieving fast error convergence rates on highly regular learning problems. To resolve saturation, we introduce iterated regularization in density ratio estimation to achieve fast error rates. Our methods outperform its non-iteratively regularized versions on benchmarks for density ratio estimation as well as on large-scale evaluations for importance-weighted ensembling of deep unsupervised domain adaptation models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13891 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Overcoming Saturation in Density Ratio Estimation by Iterated Regularization Gruber, Lukas Holzleitner, Markus Lehner, Johannes Hochreiter, Sepp Zellinger, Werner Machine Learning Estimating the ratio of two probability densities from finitely many samples, is a central task in machine learning and statistics. In this work, we show that a large class of kernel methods for density ratio estimation suffers from error saturation, which prevents algorithms from achieving fast error convergence rates on highly regular learning problems. To resolve saturation, we introduce iterated regularization in density ratio estimation to achieve fast error rates. Our methods outperform its non-iteratively regularized versions on benchmarks for density ratio estimation as well as on large-scale evaluations for importance-weighted ensembling of deep unsupervised domain adaptation models. |
| title | Overcoming Saturation in Density Ratio Estimation by Iterated Regularization |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2402.13891 |