AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911285196619776 |
|---|---|
| author | Belucci, Bruno Lounici, Karim Meziani, Katia |
| author_facet | Belucci, Bruno Lounici, Karim Meziani, Katia |
| contents | Neural networks struggle on small tabular datasets, where tree-based models remain dominant. We introduce Adaptive Contrastive Approach (AdaCap), a training scheme that combines a permutation-based contrastive loss with a Tikhonov-based closed-form output mapping. Across 85 real-world regression datasets and multiple architectures, AdaCap yields consistent and statistically significant improvements in the small-sample regime, particularly for residual models. A meta-predictor trained on dataset characteristics (size, skewness, noise) accurately anticipates when AdaCap is beneficial. These results show that AdaCap acts as a targeted regularization mechanism, strengthening neural networks precisely where they are most fragile. All results and code are publicly available at https://github.com/BrunoBelucci/adacap. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_20170 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks Belucci, Bruno Lounici, Karim Meziani, Katia Machine Learning I.5.1; I.2.6 Neural networks struggle on small tabular datasets, where tree-based models remain dominant. We introduce Adaptive Contrastive Approach (AdaCap), a training scheme that combines a permutation-based contrastive loss with a Tikhonov-based closed-form output mapping. Across 85 real-world regression datasets and multiple architectures, AdaCap yields consistent and statistically significant improvements in the small-sample regime, particularly for residual models. A meta-predictor trained on dataset characteristics (size, skewness, noise) accurately anticipates when AdaCap is beneficial. These results show that AdaCap acts as a targeted regularization mechanism, strengthening neural networks precisely where they are most fragile. All results and code are publicly available at https://github.com/BrunoBelucci/adacap. |
| title | AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks |
| topic | Machine Learning I.5.1; I.2.6 |
| url | https://arxiv.org/abs/2511.20170 |