Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.07998 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917951033049088 |
|---|---|
| author | Kong, Hangyang Zhou, Wenbo He, Xuxiang Tu, Xiaotong Ding, Xinghao |
| author_facet | Kong, Hangyang Zhou, Wenbo He, Xuxiang Tu, Xiaotong Ding, Xinghao |
| contents | Huge amount of data is the key of the success of deep learning, however, redundant information impairs the generalization ability of the model and increases the burden of calculation. Dataset Distillation (DD) compresses the original dataset into a smaller but representative subset for high-quality data and efficient training strategies. Existing works for DD generate synthetic images by treating each image as an independent entity, thereby overlooking the common features among data. This paper proposes a dataset distillation method based on Matching Training Trajectories with Low-rank Space Sampling(MTT-LSS), which uses low-rank approximations to capture multiple low-dimensional manifold subspaces of the original data. The synthetic data is represented by basis vectors and shared dimension mappers from these subspaces, reducing the cost of generating individual data points while effectively minimizing information redundancy. The proposed method is tested on CIFAR-10, CIFAR-100, and SVHN datasets, and outperforms the baseline methods by an average of 9.9%. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_07998 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Efficient Dataset Distillation through Low-Rank Space Sampling Kong, Hangyang Zhou, Wenbo He, Xuxiang Tu, Xiaotong Ding, Xinghao Computer Vision and Pattern Recognition Huge amount of data is the key of the success of deep learning, however, redundant information impairs the generalization ability of the model and increases the burden of calculation. Dataset Distillation (DD) compresses the original dataset into a smaller but representative subset for high-quality data and efficient training strategies. Existing works for DD generate synthetic images by treating each image as an independent entity, thereby overlooking the common features among data. This paper proposes a dataset distillation method based on Matching Training Trajectories with Low-rank Space Sampling(MTT-LSS), which uses low-rank approximations to capture multiple low-dimensional manifold subspaces of the original data. The synthetic data is represented by basis vectors and shared dimension mappers from these subspaces, reducing the cost of generating individual data points while effectively minimizing information redundancy. The proposed method is tested on CIFAR-10, CIFAR-100, and SVHN datasets, and outperforms the baseline methods by an average of 9.9%. |
| title | Efficient Dataset Distillation through Low-Rank Space Sampling |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.07998 |