Joint Model and Data Sparsification via the Marginal Likelihood
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866911728372023296 |
|---|---|
| author | Timans, Alexander Möllenhoff, Thomas Naesseth, Christian A. Khan, Mohammad Emtiyaz Nalisnick, Eric |
| author_facet | Timans, Alexander Möllenhoff, Thomas Naesseth, Christian A. Khan, Mohammad Emtiyaz Nalisnick, Eric |
| contents | Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic relevance determination (ARD), offers a practical Bayesian mechanism for feature sparsity via marginal likelihood optimization. Yet, its reliance on a homoscedastic noise model renders it sensitive to data contaminations such as outliers or misspecified noise, harming model fit and predictions. Instead, we propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification via a single Bayesian objective. This symmetric pruning of model and data offers a natural extension that preserves conjugacy, admits closed-form updates for standard optimization procedures, and aligns with perspectives from robust regression and influence functions. Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_29908 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Joint Model and Data Sparsification via the Marginal Likelihood Timans, Alexander Möllenhoff, Thomas Naesseth, Christian A. Khan, Mohammad Emtiyaz Nalisnick, Eric Machine Learning Sparse recovery in linear systems underpins applications from signal processing to high-dimensional regression. Sparse Bayesian Learning, grounded in the principle of automatic relevance determination (ARD), offers a practical Bayesian mechanism for feature sparsity via marginal likelihood optimization. Yet, its reliance on a homoscedastic noise model renders it sensitive to data contaminations such as outliers or misspecified noise, harming model fit and predictions. Instead, we propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification via a single Bayesian objective. This symmetric pruning of model and data offers a natural extension that preserves conjugacy, admits closed-form updates for standard optimization procedures, and aligns with perspectives from robust regression and influence functions. Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models. |
| title | Joint Model and Data Sparsification via the Marginal Likelihood |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.29908 |