Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914621195026432 |
|---|---|
| author | Stein, Viktor Datar, Adwait Ay, Nihat |
| author_facet | Stein, Viktor Datar, Adwait Ay, Nihat |
| contents | Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes. Using a unified information-geometric framework, we introduce KL analogs by replacing the Fisher-Rao geometry in the dynamical formulation of the KL with transport-based geometries, and derive closed-form expressions for common distribution families. Between elliptic distributions, these divergences remain finite for degenerating equal covariances and yield a geometric interpretation of regularization heuristics used in Kalman ensemble methods. We demonstrate the utility of these divergences in KL-regularized optimal control. In the fully tractable setting of linear time-invariant systems with Gaussian process noise, the classical KL reduces to a quadratic control penalty that becomes singular as process noise vanishes. Our variants remove this singularity and yield well-posed problems. In both the double integrator and cart-pole examples, the resulting controls preserve nontrivial feedback and achieve better closed-loop performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02250 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences Stein, Viktor Datar, Adwait Ay, Nihat Optimization and Control Machine Learning 93E20 (Primary) 49N10, 49Q22, 53B12, 68T05 (Secondary) Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes. Using a unified information-geometric framework, we introduce KL analogs by replacing the Fisher-Rao geometry in the dynamical formulation of the KL with transport-based geometries, and derive closed-form expressions for common distribution families. Between elliptic distributions, these divergences remain finite for degenerating equal covariances and yield a geometric interpretation of regularization heuristics used in Kalman ensemble methods. We demonstrate the utility of these divergences in KL-regularized optimal control. In the fully tractable setting of linear time-invariant systems with Gaussian process noise, the classical KL reduces to a quadratic control penalty that becomes singular as process noise vanishes. Our variants remove this singularity and yield well-posed problems. In both the double integrator and cart-pole examples, the resulting controls preserve nontrivial feedback and achieve better closed-loop performance. |
| title | Well-Posed KL-Regularized Control via Wasserstein and Kalman-Wasserstein KL Divergences |
| topic | Optimization and Control Machine Learning 93E20 (Primary) 49N10, 49Q22, 53B12, 68T05 (Secondary) |
| url | https://arxiv.org/abs/2602.02250 |