Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2205.13503 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918177920778240 |
|---|---|
| author | Daniels, Mara Gerbelot, Cédric Krzakala, Florent Zdeborová, Lenka |
| author_facet | Daniels, Mara Gerbelot, Cédric Krzakala, Florent Zdeborová, Lenka |
| contents | Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generative priors with fully connected layers and Gaussian i.i.d. weights, this was achieved by the multi-layer approximate message (ML-AMP) algorithm via a rigorous state evolution. However, practical generative priors are typically convolutional, allowing for computational benefits and inductive biases, and so the Gaussian i.i.d. weight assumption is very limiting. In this paper, we overcome this limitation and establish the state evolution of ML-AMP for random convolutional layers. We prove in particular that random convolutional layers belong to the same universality class as Gaussian matrices. Our proof technique is of an independent interest as it establishes a mapping between convolutional matrices and spatially coupled sensing matrices used in coding theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2205_13503 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Multi-layer State Evolution Under Random Convolutional Design Daniels, Mara Gerbelot, Cédric Krzakala, Florent Zdeborová, Lenka Information Theory Signal recovery under generative neural network priors has emerged as a promising direction in statistical inference and computational imaging. Theoretical analysis of reconstruction algorithms under generative priors is, however, challenging. For generative priors with fully connected layers and Gaussian i.i.d. weights, this was achieved by the multi-layer approximate message (ML-AMP) algorithm via a rigorous state evolution. However, practical generative priors are typically convolutional, allowing for computational benefits and inductive biases, and so the Gaussian i.i.d. weight assumption is very limiting. In this paper, we overcome this limitation and establish the state evolution of ML-AMP for random convolutional layers. We prove in particular that random convolutional layers belong to the same universality class as Gaussian matrices. Our proof technique is of an independent interest as it establishes a mapping between convolutional matrices and spatially coupled sensing matrices used in coding theory. |
| title | Multi-layer State Evolution Under Random Convolutional Design |
| topic | Information Theory |
| url | https://arxiv.org/abs/2205.13503 |