Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908669637033984 |
|---|---|
| author | Jiang, Chenyang Zhao, Hang Zhang, Xinyu Li, Zhengcen Shan, Qiben Wu, Shaocong Su, Jingyong |
| author_facet | Jiang, Chenyang Zhao, Hang Zhang, Xinyu Li, Zhengcen Shan, Qiben Wu, Shaocong Su, Jingyong |
| contents | Dataset distillation compresses large-scale datasets into compact, highly informative synthetic data, significantly reducing storage and training costs. However, existing research primarily focuses on balanced datasets and struggles to perform under real-world long-tailed distributions. In this work, we emphasize the critical role of soft labels in long-tailed dataset distillation and uncover the underlying mechanisms contributing to performance degradation. Specifically, we derive an imbalance-aware generalization bound for model trained on distilled dataset. We then identify two primary sources of soft-label bias, which originate from the distillation model and the distilled images, through systematic perturbation of the data imbalance levels. To address this, we propose ADSA, an Adaptive Soft-label Alignment module that calibrates the entangled biases. This lightweight module integrates seamlessly into existing distillation pipelines and consistently improves performance. On ImageNet-1k-LT with EDC and IPC=50, ADSA improves tail-class accuracy by up to 11.8% and raises overall accuracy to 41.4%. Extensive experiments demonstrate that ADSA provides a robust and generalizable solution under limited label budgets and across a range of distillation techniques. Code is available at: https://github.com/j-cyoung/ADSA_DD.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_17914 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation Jiang, Chenyang Zhao, Hang Zhang, Xinyu Li, Zhengcen Shan, Qiben Wu, Shaocong Su, Jingyong Computer Vision and Pattern Recognition Artificial Intelligence I.2 Dataset distillation compresses large-scale datasets into compact, highly informative synthetic data, significantly reducing storage and training costs. However, existing research primarily focuses on balanced datasets and struggles to perform under real-world long-tailed distributions. In this work, we emphasize the critical role of soft labels in long-tailed dataset distillation and uncover the underlying mechanisms contributing to performance degradation. Specifically, we derive an imbalance-aware generalization bound for model trained on distilled dataset. We then identify two primary sources of soft-label bias, which originate from the distillation model and the distilled images, through systematic perturbation of the data imbalance levels. To address this, we propose ADSA, an Adaptive Soft-label Alignment module that calibrates the entangled biases. This lightweight module integrates seamlessly into existing distillation pipelines and consistently improves performance. On ImageNet-1k-LT with EDC and IPC=50, ADSA improves tail-class accuracy by up to 11.8% and raises overall accuracy to 41.4%. Extensive experiments demonstrate that ADSA provides a robust and generalizable solution under limited label budgets and across a range of distillation techniques. Code is available at: https://github.com/j-cyoung/ADSA_DD.git. |
| title | Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence I.2 |
| url | https://arxiv.org/abs/2511.17914 |