Interpolating between Optimal Transport and KL regularized Optimal Transport using Rényi Divergences
Fuente:
arXiv
Guardado en:
| Autores principales: | , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917040743251968 |
|---|---|
| author | Bresch, Jonas Stein, Viktor |
| author_facet | Bresch, Jonas Stein, Viktor |
| contents | Regularized optimal transport (OT) has received much attention in recent years starting from Cuturi's introduction of Kullback-Leibler (KL) divergence regularized OT. In this paper, we propose regularizing the OT problem using the family of $α$-Rényi divergences for $α\in (0, 1)$. Rényi divergences are neither $f$-divergences nor Bregman distances, but they recover the KL divergence in the limit $α\nearrow 1$. The advantage of introducing the additional parameter $α$ is that for $α\searrow 0$ we obtain convergence to the unregularized OT problem. For the KL regularized OT problem, this was achieved by letting the regularization parameter $\varepsilon$ tend to zero, which causes numerical instabilities. We present two different ways to obtain premetrics on probability measures, namely by Rényi divergence constraints and by penalization. The latter premetric interpolates between the unregularized and the KL regularized OT problem with weak convergence of the unique minimizer, generalizing the interpolation property of KL regularized OT. We use a nested mirror descent algorithm to solve the primal formulation. Both on real and synthetic data sets Rényi regularized OT plans outperform their KL and Tsallis counterparts in terms of being closer to the unregularized transport plans and recovering the ground truth in inference tasks better. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_18834 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Interpolating between Optimal Transport and KL regularized Optimal Transport using Rényi Divergences Bresch, Jonas Stein, Viktor Optimization and Control Numerical Analysis Functional Analysis 49Q22, 46N10, 94A15 Regularized optimal transport (OT) has received much attention in recent years starting from Cuturi's introduction of Kullback-Leibler (KL) divergence regularized OT. In this paper, we propose regularizing the OT problem using the family of $α$-Rényi divergences for $α\in (0, 1)$. Rényi divergences are neither $f$-divergences nor Bregman distances, but they recover the KL divergence in the limit $α\nearrow 1$. The advantage of introducing the additional parameter $α$ is that for $α\searrow 0$ we obtain convergence to the unregularized OT problem. For the KL regularized OT problem, this was achieved by letting the regularization parameter $\varepsilon$ tend to zero, which causes numerical instabilities. We present two different ways to obtain premetrics on probability measures, namely by Rényi divergence constraints and by penalization. The latter premetric interpolates between the unregularized and the KL regularized OT problem with weak convergence of the unique minimizer, generalizing the interpolation property of KL regularized OT. We use a nested mirror descent algorithm to solve the primal formulation. Both on real and synthetic data sets Rényi regularized OT plans outperform their KL and Tsallis counterparts in terms of being closer to the unregularized transport plans and recovering the ground truth in inference tasks better. |
| title | Interpolating between Optimal Transport and KL regularized Optimal Transport using Rényi Divergences |
| topic | Optimization and Control Numerical Analysis Functional Analysis 49Q22, 46N10, 94A15 |
| url | https://arxiv.org/abs/2404.18834 |