Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2502.12607 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913695879135232 |
|---|---|
| author | Aoyama, Tatsuya Yang, Hanting Hanada, Hiroyuki Akahane, Satoshi Tanaka, Tomonari Okura, Yoshito Inatsu, Yu Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro |
| author_facet | Aoyama, Tatsuya Yang, Hanting Hanada, Hiroyuki Akahane, Satoshi Tanaka, Tomonari Okura, Yoshito Inatsu, Yu Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro |
| contents | We propose Duality Gap KIP (DGKIP), an extension of the Kernel Inducing Points (KIP) method for dataset distillation. While existing dataset distillation methods often rely on bi-level optimization, DGKIP eliminates the need for such optimization by leveraging duality theory in convex programming. The KIP method has been introduced as a way to avoid bi-level optimization; however, it is limited to the squared loss and does not support other loss functions (e.g., cross-entropy or hinge loss) that are more suitable for classification tasks. DGKIP addresses this limitation by exploiting an upper bound on parameter changes after dataset distillation using the duality gap, enabling its application to a wider range of loss functions. We also characterize theoretical properties of DGKIP by providing upper bounds on the test error and prediction consistency after dataset distillation. Experimental results on standard benchmarks such as MNIST and CIFAR-10 demonstrate that DGKIP retains the efficiency of KIP while offering broader applicability and robust performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_12607 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generalized Kernel Inducing Points by Duality Gap for Dataset Distillation Aoyama, Tatsuya Yang, Hanting Hanada, Hiroyuki Akahane, Satoshi Tanaka, Tomonari Okura, Yoshito Inatsu, Yu Hashimoto, Noriaki Murayama, Taro Lee, Hanju Kojima, Shinya Takeuchi, Ichiro Machine Learning We propose Duality Gap KIP (DGKIP), an extension of the Kernel Inducing Points (KIP) method for dataset distillation. While existing dataset distillation methods often rely on bi-level optimization, DGKIP eliminates the need for such optimization by leveraging duality theory in convex programming. The KIP method has been introduced as a way to avoid bi-level optimization; however, it is limited to the squared loss and does not support other loss functions (e.g., cross-entropy or hinge loss) that are more suitable for classification tasks. DGKIP addresses this limitation by exploiting an upper bound on parameter changes after dataset distillation using the duality gap, enabling its application to a wider range of loss functions. We also characterize theoretical properties of DGKIP by providing upper bounds on the test error and prediction consistency after dataset distillation. Experimental results on standard benchmarks such as MNIST and CIFAR-10 demonstrate that DGKIP retains the efficiency of KIP while offering broader applicability and robust performance. |
| title | Generalized Kernel Inducing Points by Duality Gap for Dataset Distillation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.12607 |