Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.00407 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908387929751552 |
|---|---|
| author | Chen, Ruixuan Li, Wentao Xiao, Jiahui Li, Yuchen Tang, Yimin Wang, Xiaonan |
| author_facet | Chen, Ruixuan Li, Wentao Xiao, Jiahui Li, Yuchen Tang, Yimin Wang, Xiaonan |
| contents | Machine learning models often degrade when deployed on data distributions different from their training data. Challenging conventional validation paradigms, we demonstrate that higher in-distribution (ID) bias can lead to better out-of-distribution (OOD) generalization. Our Adaptive Distribution Bridge (ADB) framework implements this insight by introducing controlled statistical diversity during training, enabling models to develop bias profiles that effectively generalize across distributions. Empirically, we observe a robust negative correlation where higher ID bias corresponds to lower OOD error--a finding that contradicts standard practices focused on minimizing validation error. Evaluation on multiple datasets shows our approach significantly improves OOD generalization. ADB achieves robust mean error reductions of up to 26.8% compared to traditional cross-validation, and consistently identifies high-performing training strategies, evidenced by percentile ranks often exceeding 74.4%. Our work provides both a practical method for improving generalization and a theoretical framework for reconsidering the role of bias in robust machine learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_00407 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Bias as a Virtue: Rethinking Generalization under Distribution Shifts Chen, Ruixuan Li, Wentao Xiao, Jiahui Li, Yuchen Tang, Yimin Wang, Xiaonan Machine Learning Artificial Intelligence Machine learning models often degrade when deployed on data distributions different from their training data. Challenging conventional validation paradigms, we demonstrate that higher in-distribution (ID) bias can lead to better out-of-distribution (OOD) generalization. Our Adaptive Distribution Bridge (ADB) framework implements this insight by introducing controlled statistical diversity during training, enabling models to develop bias profiles that effectively generalize across distributions. Empirically, we observe a robust negative correlation where higher ID bias corresponds to lower OOD error--a finding that contradicts standard practices focused on minimizing validation error. Evaluation on multiple datasets shows our approach significantly improves OOD generalization. ADB achieves robust mean error reductions of up to 26.8% compared to traditional cross-validation, and consistently identifies high-performing training strategies, evidenced by percentile ranks often exceeding 74.4%. Our work provides both a practical method for improving generalization and a theoretical framework for reconsidering the role of bias in robust machine learning. |
| title | Bias as a Virtue: Rethinking Generalization under Distribution Shifts |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.00407 |