On the Anisotropy of Score-Based Generative Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915579558887424 |
|---|---|
| author | Floros, Andreas Moosavi-Dezfooli, Seyed-Mohsen Dragotti, Pier Luigi |
| author_facet | Floros, Andreas Moosavi-Dezfooli, Seyed-Mohsen Dragotti, Pier Luigi |
| contents | We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis shows that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_22899 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Anisotropy of Score-Based Generative Models Floros, Andreas Moosavi-Dezfooli, Seyed-Mohsen Dragotti, Pier Luigi Machine Learning We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis shows that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models. |
| title | On the Anisotropy of Score-Based Generative Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2510.22899 |