Activation-Space Uncertainty Quantification for Pretrained Networks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912919442161664 |
|---|---|
| author | Bergna, Richard Depeweg, Stefan Calvo-Ordoñez, Sergio Plenk, Jonathan Cartea, Alvaro Hernández-Lobato, Jose Miguel |
| author_facet | Bergna, Richard Depeweg, Stefan Calvo-Ordoñez, Sergio Plenk, Jonathan Cartea, Alvaro Hernández-Lobato, Jose Miguel |
| contents | Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone's predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form epistemic variances in activation space. To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation without sampling, backpropagation, or second-order information. Across regression, classification, image segmentation, and language modeling, GAPA matches or outperforms strong post-hoc baselines in calibration and out-of-distribution detection while remaining efficient at test time. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_14934 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Activation-Space Uncertainty Quantification for Pretrained Networks Bergna, Richard Depeweg, Stefan Calvo-Ordoñez, Sergio Plenk, Jonathan Cartea, Alvaro Hernández-Lobato, Jose Miguel Machine Learning Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone's predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form epistemic variances in activation space. To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation without sampling, backpropagation, or second-order information. Across regression, classification, image segmentation, and language modeling, GAPA matches or outperforms strong post-hoc baselines in calibration and out-of-distribution detection while remaining efficient at test time. |
| title | Activation-Space Uncertainty Quantification for Pretrained Networks |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.14934 |