Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2309.13332 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912065164148736 |
|---|---|
| author | Lacker, Daniel |
| author_facet | Lacker, Daniel |
| contents | What is the optimal way to approximate a high-dimensional diffusion process by one in which the coordinates are independent? This paper presents a construction, called the \emph{independent projection}, which is optimal for two natural criteria. First, when the original diffusion is reversible with invariant measure $ρ_*$, the independent projection serves as the Wasserstein gradient flow for the relative entropy $H(\cdot\,|\,ρ_*)$ constrained to the space of product measures. This is related to recent Langevin-based sampling schemes proposed in the statistical literature on mean field variational inference. In addition, we provide both qualitative and quantitative results on the long-time convergence of the independent projection, with quantitative results in the log-concave case derived via a new variant of the logarithmic Sobolev inequality. Second, among all processes with independent coordinates, the independent projection is shown to exhibit the slowest growth rate of path-space entropy relative to the original diffusion. This sheds new light on the classical McKean-Vlasov equation and recent variants proposed for non-exchangeable systems, which can be viewed as special cases of the independent projection. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2309_13332 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Independent projections of diffusions: Gradient flows for variational inference and optimal mean field approximations Lacker, Daniel Probability Analysis of PDEs Machine Learning What is the optimal way to approximate a high-dimensional diffusion process by one in which the coordinates are independent? This paper presents a construction, called the \emph{independent projection}, which is optimal for two natural criteria. First, when the original diffusion is reversible with invariant measure $ρ_*$, the independent projection serves as the Wasserstein gradient flow for the relative entropy $H(\cdot\,|\,ρ_*)$ constrained to the space of product measures. This is related to recent Langevin-based sampling schemes proposed in the statistical literature on mean field variational inference. In addition, we provide both qualitative and quantitative results on the long-time convergence of the independent projection, with quantitative results in the log-concave case derived via a new variant of the logarithmic Sobolev inequality. Second, among all processes with independent coordinates, the independent projection is shown to exhibit the slowest growth rate of path-space entropy relative to the original diffusion. This sheds new light on the classical McKean-Vlasov equation and recent variants proposed for non-exchangeable systems, which can be viewed as special cases of the independent projection. |
| title | Independent projections of diffusions: Gradient flows for variational inference and optimal mean field approximations |
| topic | Probability Analysis of PDEs Machine Learning |
| url | https://arxiv.org/abs/2309.13332 |