Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.04671 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915997245505536 |
|---|---|
| author | Su, Ye Zhao, Longlong Garcia-Gil, Diego Guo, Jipeng Zhang, Gangchun Chen, Jinxin Chen, Jinsong |
| author_facet | Su, Ye Zhao, Longlong Garcia-Gil, Diego Guo, Jipeng Zhang, Gangchun Chen, Jinxin Chen, Jinsong |
| contents | Gradient boosting remains a strong and widely used method for tabular data learning, but its performance often degrades when training labels are noisy. This behavior is largely related to the way boosting algorithms emphasize samples with large gradients, without explicitly accounting for whether such errors originate from informative hard cases or from unreliable labels. We address this issue by reconsidering how sample reliability is evaluated during boosting. Instead of relying on instantaneous error, we examine the evolution of each sample's residuals across iterations. Based on this insight, we propose Information-Theoretic Trust Boosting (ITBoost), which uses the Minimum Description Length principle to measure the complexity of residual trajectories. Samples whose residual patterns fluctuate in an irregular manner are treated as less trustworthy and are down-weighted during learning. Theoretically, we derive a tighter generalization bound for ITBoost under label noise. Empirical results on various tabular benchmarks indicate that ITBoost provides improved robustness in noisy environments over leading boosting and deep tabular models, while retaining best average performance on clean data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_04671 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ITBoost: Information-Theoretic Trust for Robust Boosting Su, Ye Zhao, Longlong Garcia-Gil, Diego Guo, Jipeng Zhang, Gangchun Chen, Jinxin Chen, Jinsong Machine Learning Gradient boosting remains a strong and widely used method for tabular data learning, but its performance often degrades when training labels are noisy. This behavior is largely related to the way boosting algorithms emphasize samples with large gradients, without explicitly accounting for whether such errors originate from informative hard cases or from unreliable labels. We address this issue by reconsidering how sample reliability is evaluated during boosting. Instead of relying on instantaneous error, we examine the evolution of each sample's residuals across iterations. Based on this insight, we propose Information-Theoretic Trust Boosting (ITBoost), which uses the Minimum Description Length principle to measure the complexity of residual trajectories. Samples whose residual patterns fluctuate in an irregular manner are treated as less trustworthy and are down-weighted during learning. Theoretically, we derive a tighter generalization bound for ITBoost under label noise. Empirical results on various tabular benchmarks indicate that ITBoost provides improved robustness in noisy environments over leading boosting and deep tabular models, while retaining best average performance on clean data. |
| title | ITBoost: Information-Theoretic Trust for Robust Boosting |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2605.04671 |