C2G-KD: PCA-Constrained Generator for Data-Free Knowledge Distillation
Fuente:
arXiv
Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909703856979968 |
|---|---|
| author | Bengtsson, Magnus Östberg, Kenneth |
| author_facet | Bengtsson, Magnus Östberg, Kenneth |
| contents | We introduce C2G-KD, a data-free knowledge distillation framework where a class-conditional generator is trained to produce synthetic samples guided by a frozen teacher model and geometric constraints derived from PCA. The generator never observes real training data but instead learns to activate the teacher's output through a combination of semantic and structural losses. By constraining generated samples to lie within class-specific PCA subspaces estimated from as few as two real examples per class, we preserve topological consistency and diversity. Experiments on MNIST show that even minimal class structure is sufficient to bootstrap useful synthetic training pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_18533 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | C2G-KD: PCA-Constrained Generator for Data-Free Knowledge Distillation Bengtsson, Magnus Östberg, Kenneth Machine Learning Artificial Intelligence We introduce C2G-KD, a data-free knowledge distillation framework where a class-conditional generator is trained to produce synthetic samples guided by a frozen teacher model and geometric constraints derived from PCA. The generator never observes real training data but instead learns to activate the teacher's output through a combination of semantic and structural losses. By constraining generated samples to lie within class-specific PCA subspaces estimated from as few as two real examples per class, we preserve topological consistency and diversity. Experiments on MNIST show that even minimal class structure is sufficient to bootstrap useful synthetic training pipelines. |
| title | C2G-KD: PCA-Constrained Generator for Data-Free Knowledge Distillation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2507.18533 |