Information-theoretic limits and approximate message-passing for high-dimensional time series
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909542063800320 |
|---|---|
| author | Tieplova, Daria Lahiry, Samriddha Barbier, Jean |
| author_facet | Tieplova, Daria Lahiry, Samriddha Barbier, Jean |
| contents | High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. Theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signal. In non-sparse regimes, analyses have primarily focused on linear regression models with the design matrix having independent rows. In this paper, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal, as well as for minimum mean-square errors. We also empirically study the vector approximate message passing VAMP algorithm and show that, despite the lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_13625 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Information-theoretic limits and approximate message-passing for high-dimensional time series Tieplova, Daria Lahiry, Samriddha Barbier, Jean Information Theory Disordered Systems and Neural Networks Statistics Theory 82Bxx, 62Mxx High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. Theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signal. In non-sparse regimes, analyses have primarily focused on linear regression models with the design matrix having independent rows. In this paper, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal, as well as for minimum mean-square errors. We also empirically study the vector approximate message passing VAMP algorithm and show that, despite the lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal. |
| title | Information-theoretic limits and approximate message-passing for high-dimensional time series |
| topic | Information Theory Disordered Systems and Neural Networks Statistics Theory 82Bxx, 62Mxx |
| url | https://arxiv.org/abs/2501.13625 |