Poincare Inequality for Local Log-Polyak-Łojasiewicz Measures: Non-asymptotic Analysis in Low-temperature Regime
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915160020484096 |
|---|---|
| author | Gong, Yun He, Niao Shen, Zebang |
| author_facet | Gong, Yun He, Niao Shen, Zebang |
| contents | Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such landscapes, we propose to study the class of log-PŁ$^\circ$ measures $μ_ε\propto \exp(-V/ε)$, where the potential $V$ satisfies a local Polyak-Łojasiewicz (PŁ) inequality, and its set of local minima is provably connected. Notably, potentials in this class can exhibit local maxima and we characterize its optimal set $S$ to be a compact ${C}^2$ embedding submanifold of ${R}^d$ without boundary. The non-contractibility of $S$ distinguishes our function class from the classical convex setting topologically. Moreover, the embedding structure induces a naturally defined Laplacian-Beltrami operator on $S$, and we show that its first non-trivial eigenvalue provides an $ε$-independent lower bound for the Poincaré constant in the Poincaré inequality of $μ_ε$. As a direct consequence, Langevin dynamics with such non-convex potential $V$ and diffusion coefficient $ε$ converges to its equilibrium $μ_ε$ at a rate of $\tilde{O}(1/ε)$, provided $ε$ is sufficiently small. Here $\tilde{O}$ hides logarithmic terms. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_00429 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Poincare Inequality for Local Log-Polyak-Łojasiewicz Measures: Non-asymptotic Analysis in Low-temperature Regime Gong, Yun He, Niao Shen, Zebang Probability Functional Analysis Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such landscapes, we propose to study the class of log-PŁ$^\circ$ measures $μ_ε\propto \exp(-V/ε)$, where the potential $V$ satisfies a local Polyak-Łojasiewicz (PŁ) inequality, and its set of local minima is provably connected. Notably, potentials in this class can exhibit local maxima and we characterize its optimal set $S$ to be a compact ${C}^2$ embedding submanifold of ${R}^d$ without boundary. The non-contractibility of $S$ distinguishes our function class from the classical convex setting topologically. Moreover, the embedding structure induces a naturally defined Laplacian-Beltrami operator on $S$, and we show that its first non-trivial eigenvalue provides an $ε$-independent lower bound for the Poincaré constant in the Poincaré inequality of $μ_ε$. As a direct consequence, Langevin dynamics with such non-convex potential $V$ and diffusion coefficient $ε$ converges to its equilibrium $μ_ε$ at a rate of $\tilde{O}(1/ε)$, provided $ε$ is sufficiently small. Here $\tilde{O}$ hides logarithmic terms. |
| title | Poincare Inequality for Local Log-Polyak-Łojasiewicz Measures: Non-asymptotic Analysis in Low-temperature Regime |
| topic | Probability Functional Analysis |
| url | https://arxiv.org/abs/2501.00429 |