Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911948035063808 |
|---|---|
| author | Barbier, Jean Camilli, Francesco Mondelli, Marco Xu, Yizhou |
| author_facet | Barbier, Jean Camilli, Francesco Mondelli, Marco Xu, Yizhou |
| contents | We consider a prototypical problem of Bayesian inference for a structured spiked model: a low-rank signal is corrupted by additive noise. While both information-theoretic and algorithmic limits are well understood when the noise is a Gaussian Wigner matrix, the more realistic case of structured noise still proves to be challenging. To capture the structure while maintaining mathematical tractability, a line of work has focused on rotationally invariant noise. However, existing studies either provide sub-optimal algorithms or are limited to special cases of noise ensembles. In this paper, using tools from statistical physics (replica method) and random matrix theory (generalized spherical integrals) we establish the first characterization of the information-theoretic limits for a noise matrix drawn from a general trace ensemble. Remarkably, our analysis unveils the asymptotic equivalence between the rotationally invariant model and a surrogate Gaussian one. Finally, we show how to saturate the predicted statistical limits using an efficient algorithm inspired by the theory of adaptive Thouless-Anderson-Palmer (TAP) equations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_20993 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise Barbier, Jean Camilli, Francesco Mondelli, Marco Xu, Yizhou Information Theory Disordered Systems and Neural Networks Machine Learning Statistics Theory 62F15, 82B44 We consider a prototypical problem of Bayesian inference for a structured spiked model: a low-rank signal is corrupted by additive noise. While both information-theoretic and algorithmic limits are well understood when the noise is a Gaussian Wigner matrix, the more realistic case of structured noise still proves to be challenging. To capture the structure while maintaining mathematical tractability, a line of work has focused on rotationally invariant noise. However, existing studies either provide sub-optimal algorithms or are limited to special cases of noise ensembles. In this paper, using tools from statistical physics (replica method) and random matrix theory (generalized spherical integrals) we establish the first characterization of the information-theoretic limits for a noise matrix drawn from a general trace ensemble. Remarkably, our analysis unveils the asymptotic equivalence between the rotationally invariant model and a surrogate Gaussian one. Finally, we show how to saturate the predicted statistical limits using an efficient algorithm inspired by the theory of adaptive Thouless-Anderson-Palmer (TAP) equations. |
| title | Information limits and Thouless-Anderson-Palmer equations for spiked matrix models with structured noise |
| topic | Information Theory Disordered Systems and Neural Networks Machine Learning Statistics Theory 62F15, 82B44 |
| url | https://arxiv.org/abs/2405.20993 |