Richer Bayesian Last Layers with Subsampled NTK Features
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913150701404160 |
|---|---|
| author | Calvo-Ordoñez, Sergio Plenk, Jonathan Bergna, Richard Cartea, Álvaro Gal, Yarin Hernández-Lobato, Jose Miguel Ciosek, Kamil |
| author_facet | Calvo-Ordoñez, Sergio Plenk, Jonathan Bergna, Richard Cartea, Álvaro Gal, Yarin Hernández-Lobato, Jose Miguel Ciosek, Kamil |
| contents | Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty because they apply a Bayesian treatment only to the final layer, ignoring uncertainty induced by earlier layers. We propose a method that improves BLLs by leveraging a projection of Neural Tangent Kernel (NTK) features onto the space spanned by the last-layer features. This enables posterior inference that accounts for variability of the full network while retaining the low computational cost of inference of a standard BLL. We show that our method yields posterior variances that are provably greater or equal to those of a standard BLL, correcting its tendency to underestimate epistemic uncertainty. To further reduce computational cost, we introduce a uniform subsampling scheme for estimating the projection matrix and for posterior inference. We derive approximation bounds for both types of subsampling. Empirical evaluations on UCI regression, contextual bandits, image classification, and out-of-distribution detection tasks in image and tabular datasets, demonstrate improved calibration and uncertainty estimates compared to standard BLLs and competitive baselines, while reducing computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_01279 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Richer Bayesian Last Layers with Subsampled NTK Features Calvo-Ordoñez, Sergio Plenk, Jonathan Bergna, Richard Cartea, Álvaro Gal, Yarin Hernández-Lobato, Jose Miguel Ciosek, Kamil Machine Learning Bayesian Last Layers (BLLs) provide a convenient and computationally efficient way to estimate uncertainty in neural networks. However, they underestimate epistemic uncertainty because they apply a Bayesian treatment only to the final layer, ignoring uncertainty induced by earlier layers. We propose a method that improves BLLs by leveraging a projection of Neural Tangent Kernel (NTK) features onto the space spanned by the last-layer features. This enables posterior inference that accounts for variability of the full network while retaining the low computational cost of inference of a standard BLL. We show that our method yields posterior variances that are provably greater or equal to those of a standard BLL, correcting its tendency to underestimate epistemic uncertainty. To further reduce computational cost, we introduce a uniform subsampling scheme for estimating the projection matrix and for posterior inference. We derive approximation bounds for both types of subsampling. Empirical evaluations on UCI regression, contextual bandits, image classification, and out-of-distribution detection tasks in image and tabular datasets, demonstrate improved calibration and uncertainty estimates compared to standard BLLs and competitive baselines, while reducing computational cost. |
| title | Richer Bayesian Last Layers with Subsampled NTK Features |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.01279 |