VIKING: Deep variational inference with stochastic projections
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866914117571313664 |
|---|---|
| author | Fadel, Samuel G. Roy, Hrittik Krämer, Nicholas Zainchkovskyy, Yevgen Syrota, Stas Mahou, Alejandro Valverde Ek, Carl Henrik Hauberg, Søren |
| author_facet | Fadel, Samuel G. Roy, Hrittik Krämer, Nicholas Zainchkovskyy, Yevgen Syrota, Stas Mahou, Alejandro Valverde Ek, Carl Henrik Hauberg, Søren |
| contents | Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks. Where a Bayesian treatment is usually associated with high-quality predictions and uncertainties, the practical reality has been the opposite, with unstable training, poor predictive power, and subpar calibration. Building upon recent work on reparametrizations of neural networks, we propose a simple variational family that considers two independent linear subspaces of the parameter space. These represent functional changes inside and outside the support of training data. This allows us to build a fully-correlated approximate posterior reflecting the overparametrization that tunes easy-to-interpret hyperparameters. We develop scalable numerical routines that maximize the associated evidence lower bound (ELBO) and sample from the approximate posterior. Empirically, we observe state-of-the-art performance across tasks, models, and datasets compared to a wide array of baseline methods. Our results show that approximate Bayesian inference applied to deep neural networks is far from a lost cause when constructing inference mechanisms that reflect the geometry of reparametrizations. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_23684 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | VIKING: Deep variational inference with stochastic projections Fadel, Samuel G. Roy, Hrittik Krämer, Nicholas Zainchkovskyy, Yevgen Syrota, Stas Mahou, Alejandro Valverde Ek, Carl Henrik Hauberg, Søren Machine Learning Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks. Where a Bayesian treatment is usually associated with high-quality predictions and uncertainties, the practical reality has been the opposite, with unstable training, poor predictive power, and subpar calibration. Building upon recent work on reparametrizations of neural networks, we propose a simple variational family that considers two independent linear subspaces of the parameter space. These represent functional changes inside and outside the support of training data. This allows us to build a fully-correlated approximate posterior reflecting the overparametrization that tunes easy-to-interpret hyperparameters. We develop scalable numerical routines that maximize the associated evidence lower bound (ELBO) and sample from the approximate posterior. Empirically, we observe state-of-the-art performance across tasks, models, and datasets compared to a wide array of baseline methods. Our results show that approximate Bayesian inference applied to deep neural networks is far from a lost cause when constructing inference mechanisms that reflect the geometry of reparametrizations. |
| title | VIKING: Deep variational inference with stochastic projections |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.23684 |