Derivatives of entropy and the MMSE conjecture
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909224262434816 |
|---|---|
| author | Mansanarez, Paul Poly, Guillaume Swan, Yvik |
| author_facet | Mansanarez, Paul Poly, Guillaume Swan, Yvik |
| contents | We investigate the entropy $H(μ,t)$ of a probability measure $μ$ along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that $μ$ admits moments of any order, it is indeed proved in [Guo et al., 2010] that $t\mapsto H(μ,t)$ is smooth, and in [Ledoux, 2016] that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of $μ$. In the seminal contribution \cite{Led}, these algebraic expressions are derived through $Γ$-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions.
Building upon these explicit formulas we revisit the so-called "MMSE conjecture" from [Guo et al., 2010] which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in [Ledoux, 2016]. As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2311_04831 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Derivatives of entropy and the MMSE conjecture Mansanarez, Paul Poly, Guillaume Swan, Yvik Information Theory Probability We investigate the entropy $H(μ,t)$ of a probability measure $μ$ along the heat flow and more precisely we seek for closed algebraic representations of its derivatives. Provided that $μ$ admits moments of any order, it is indeed proved in [Guo et al., 2010] that $t\mapsto H(μ,t)$ is smooth, and in [Ledoux, 2016] that its derivatives at zero can be expressed into multivariate polynomials evaluated in the moments (or cumulants) of $μ$. In the seminal contribution \cite{Led}, these algebraic expressions are derived through $Γ$-calculus techniques which provide implicit recursive formulas for these polynomials. Our main contribution consists in a fine combinatorial analysis of these inductive relations and for the first time to derive closed formulas for the leading coefficients of these polynomials expressions. Building upon these explicit formulas we revisit the so-called "MMSE conjecture" from [Guo et al., 2010] which asserts that two distributions on the real line with the same entropy along the heat flow must coincide up to translation and symmetry. Our approach enables us to provide new conditions on the source distributions ensuring that the MMSE conjecture holds and to refine several criteria proved in [Ledoux, 2016]. As illustrating examples, our findings cover the cases of uniform and Rademacher distributions, for which previous results in the literature were inapplicable. |
| title | Derivatives of entropy and the MMSE conjecture |
| topic | Information Theory Probability |
| url | https://arxiv.org/abs/2311.04831 |