Probabilistic computation and uncertainty quantification with emerging covariance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914638671642624 |
|---|---|
| author | Ma, Hengyuan Qi, Yang Zhang, Li Lu, Wenlian Feng, Jianfeng |
| author_facet | Ma, Hengyuan Qi, Yang Zhang, Li Lu, Wenlian Feng, Jianfeng |
| contents | Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_19265 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Probabilistic computation and uncertainty quantification with emerging covariance Ma, Hengyuan Qi, Yang Zhang, Li Lu, Wenlian Feng, Jianfeng Machine Learning Neural and Evolutionary Computing Statistics Theory Building robust, interpretable, and secure AI system requires quantifying and representing uncertainty under a probabilistic perspective to mimic human cognitive abilities. However, probabilistic computation presents significant challenges for most conventional artificial neural network, as they are essentially implemented in a deterministic manner. In this paper, we develop an efficient probabilistic computation framework by truncating the probabilistic representation of neural activation up to its mean and covariance and construct a moment neural network that encapsulates the nonlinear coupling between the mean and covariance of the underlying stochastic network. We reveal that when only the mean but not the covariance is supervised during gradient-based learning, the unsupervised covariance spontaneously emerges from its nonlinear coupling with the mean and faithfully captures the uncertainty associated with model predictions. Our findings highlight the inherent simplicity of probabilistic computation by seamlessly incorporating uncertainty into model prediction, paving the way for integrating it into large-scale AI systems. |
| title | Probabilistic computation and uncertainty quantification with emerging covariance |
| topic | Machine Learning Neural and Evolutionary Computing Statistics Theory |
| url | https://arxiv.org/abs/2305.19265 |