Mixup Regularization: A Probabilistic Perspective
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909648096854016 |
|---|---|
| author | El-Laham, Yousef Dalmasso, Niccolò Vyetrenko, Svitlana Potluru, Vamsi K. Veloso, Manuela |
| author_facet | El-Laham, Yousef Dalmasso, Niccolò Vyetrenko, Svitlana Potluru, Vamsi K. Veloso, Manuela |
| contents | In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_13825 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Mixup Regularization: A Probabilistic Perspective El-Laham, Yousef Dalmasso, Niccolò Vyetrenko, Svitlana Potluru, Vamsi K. Veloso, Manuela Machine Learning In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants. |
| title | Mixup Regularization: A Probabilistic Perspective |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.13825 |