An Equivalence between Bayesian Priors and Penalties in Variational Inference
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2020
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913225287663616 |
|---|---|
| author | Wolinski, Pierre Charpiat, Guillaume Ollivier, Yann |
| author_facet | Wolinski, Pierre Charpiat, Guillaume Ollivier, Yann |
| contents | In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by an ad hoc regularization term that penalizes some values of the parameters. Regularization terms appear naturally in Variational Inference, a tractable way to approximate Bayesian posteriors: the loss to optimize contains a Kullback--Leibler divergence term between the approximate posterior and a Bayesian prior. We fully characterize the regularizers that can arise according to this procedure, and provide a systematic way to compute the prior corresponding to a given penalty. Such a characterization can be used to discover constraints over the penalty function, so that the overall procedure remains Bayesian. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2002_00178 |
| institution | arXiv |
| publishDate | 2020 |
| record_format | arxiv |
| spellingShingle | An Equivalence between Bayesian Priors and Penalties in Variational Inference Wolinski, Pierre Charpiat, Guillaume Ollivier, Yann Machine Learning Statistics Theory In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by an ad hoc regularization term that penalizes some values of the parameters. Regularization terms appear naturally in Variational Inference, a tractable way to approximate Bayesian posteriors: the loss to optimize contains a Kullback--Leibler divergence term between the approximate posterior and a Bayesian prior. We fully characterize the regularizers that can arise according to this procedure, and provide a systematic way to compute the prior corresponding to a given penalty. Such a characterization can be used to discover constraints over the penalty function, so that the overall procedure remains Bayesian. |
| title | An Equivalence between Bayesian Priors and Penalties in Variational Inference |
| topic | Machine Learning Statistics Theory |
| url | https://arxiv.org/abs/2002.00178 |