Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913631531171840 |
|---|---|
| author | Padilla, Carlos Misael Madrid Padilla, Oscar Hernan Madrid Kei, Yik Lun Zhang, Zhi Chen, Yanzhen |
| author_facet | Padilla, Carlos Misael Madrid Padilla, Oscar Hernan Madrid Kei, Yik Lun Zhang, Zhi Chen, Yanzhen |
| contents | This paper addresses the problems of conditional variance estimation and confidence interval construction in nonparametric regression using dense networks with the Rectified Linear Unit (ReLU) activation function. We present a residual-based framework for conditional variance estimation, deriving nonasymptotic bounds for variance estimation under both heteroscedastic and homoscedastic settings. We relax the sub-Gaussian noise assumption, allowing the proposed bounds to accommodate sub-Exponential noise and beyond. Building on this, for a ReLU neural network estimator, we derive non-asymptotic bounds for both its conditional mean and variance estimation, representing the first result for variance estimation using ReLU networks. Furthermore, we develop a ReLU network based robust bootstrap procedure (Efron, 1992) for constructing confidence intervals for the true mean that comes with a theoretical guarantee on the coverage, providing a significant advancement in uncertainty quantification and the construction of reliable confidence intervals in deep learning settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20355 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks Padilla, Carlos Misael Madrid Padilla, Oscar Hernan Madrid Kei, Yik Lun Zhang, Zhi Chen, Yanzhen Machine Learning This paper addresses the problems of conditional variance estimation and confidence interval construction in nonparametric regression using dense networks with the Rectified Linear Unit (ReLU) activation function. We present a residual-based framework for conditional variance estimation, deriving nonasymptotic bounds for variance estimation under both heteroscedastic and homoscedastic settings. We relax the sub-Gaussian noise assumption, allowing the proposed bounds to accommodate sub-Exponential noise and beyond. Building on this, for a ReLU neural network estimator, we derive non-asymptotic bounds for both its conditional mean and variance estimation, representing the first result for variance estimation using ReLU networks. Furthermore, we develop a ReLU network based robust bootstrap procedure (Efron, 1992) for constructing confidence intervals for the true mean that comes with a theoretical guarantee on the coverage, providing a significant advancement in uncertainty quantification and the construction of reliable confidence intervals in deep learning settings. |
| title | Confidence Interval Construction and Conditional Variance Estimation with Dense ReLU Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2412.20355 |