Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866908608511344640 |
|---|---|
| author | Lee, Kyeongwon Lin, Lizhen Park, Jaewoo Jeong, Seonghyun |
| author_facet | Lee, Kyeongwon Lin, Lizhen Park, Jaewoo Jeong, Seonghyun |
| contents | This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_19144 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality Lee, Kyeongwon Lin, Lizhen Park, Jaewoo Jeong, Seonghyun Machine Learning This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems. |
| title | Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2506.19144 |