PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866914340882350080 |
|---|---|
| author | Moser, Brian B. Sarode, Shalini Raue, Federico Frolov, Stanislav Adamkiewicz, Krzysztof Shanbhag, Arundhati Folz, Joachim Nauen, Tobias C. Dengel, Andreas |
| author_facet | Moser, Brian B. Sarode, Shalini Raue, Federico Frolov, Stanislav Adamkiewicz, Krzysztof Shanbhag, Arundhati Folz, Joachim Nauen, Tobias C. Dengel, Andreas |
| contents | Dataset distillation (DD) promises compact yet faithful synthetic data, but existing approaches often inherit the inductive bias of a single teacher model. As dataset size increases, this bias drives generation toward overly smooth, homogeneous samples, reducing intra-class diversity and limiting generalization. We present PRISM (PRIors from diverse Source Models), a framework that disentangles architectural priors during synthesis. PRISM decouples the logit-matching and regularization objectives, supervising them with different teacher architectures: a primary model for logits and a stochastic subset for batch-normalization (BN) alignment. On ImageNet-1K, PRISM consistently and reproducibly outperforms single-teacher methods (e.g., SRe2L) and recent multi-teacher variants (e.g., G-VBSM) at low- and mid-IPC regimes. The generated data also show significantly richer intra-class diversity, as reflected by a notable drop in cosine similarity between features. We further analyze teacher selection strategies (pre- vs. intra-distillation) and introduce a scalable cross-class batch formation scheme for fast parallel synthesis. Code will be released after the review period. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_09905 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors Moser, Brian B. Sarode, Shalini Raue, Federico Frolov, Stanislav Adamkiewicz, Krzysztof Shanbhag, Arundhati Folz, Joachim Nauen, Tobias C. Dengel, Andreas Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Dataset distillation (DD) promises compact yet faithful synthetic data, but existing approaches often inherit the inductive bias of a single teacher model. As dataset size increases, this bias drives generation toward overly smooth, homogeneous samples, reducing intra-class diversity and limiting generalization. We present PRISM (PRIors from diverse Source Models), a framework that disentangles architectural priors during synthesis. PRISM decouples the logit-matching and regularization objectives, supervising them with different teacher architectures: a primary model for logits and a stochastic subset for batch-normalization (BN) alignment. On ImageNet-1K, PRISM consistently and reproducibly outperforms single-teacher methods (e.g., SRe2L) and recent multi-teacher variants (e.g., G-VBSM) at low- and mid-IPC regimes. The generated data also show significantly richer intra-class diversity, as reflected by a notable drop in cosine similarity between features. We further analyze teacher selection strategies (pre- vs. intra-distillation) and introduce a scalable cross-class batch formation scheme for fast parallel synthesis. Code will be released after the review period. |
| title | PRISM: Diversifying Dataset Distillation by Decoupling Architectural Priors |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.09905 |