Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914084034707456 |
|---|---|
| author | Kuipers, Bart Byrman, Freek Uyterlinde, Daniel García-Castellanos, Alejandro |
| author_facet | Kuipers, Bart Byrman, Freek Uyterlinde, Daniel García-Castellanos, Alejandro |
| contents | Amortized optimization accelerates the solution of related optimization problems by learning mappings that exploit shared structure across problem instances. We explore the use of Scale Equivariant Graph Metanetworks (ScaleGMNs) for this purpose. By operating directly in weight space, ScaleGMNs enable single-shot fine-tuning of existing models, reducing the need for iterative optimization. We demonstrate the effectiveness of this approach empirically and provide a theoretical result: the gauge freedom induced by scaling symmetries is strictly smaller in convolutional neural networks than in multi-layer perceptrons. This insight helps explain the performance differences observed between architectures in both our work and that of Kalogeropoulos et al. (2024). Overall, our findings underscore the potential of symmetry-aware metanetworks as a powerful approach for efficient and generalizable neural network optimization. Open-source code: https://github.com/daniuyter/scalegmn_amortization |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_08300 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks Kuipers, Bart Byrman, Freek Uyterlinde, Daniel García-Castellanos, Alejandro Artificial Intelligence Amortized optimization accelerates the solution of related optimization problems by learning mappings that exploit shared structure across problem instances. We explore the use of Scale Equivariant Graph Metanetworks (ScaleGMNs) for this purpose. By operating directly in weight space, ScaleGMNs enable single-shot fine-tuning of existing models, reducing the need for iterative optimization. We demonstrate the effectiveness of this approach empirically and provide a theoretical result: the gauge freedom induced by scaling symmetries is strictly smaller in convolutional neural networks than in multi-layer perceptrons. This insight helps explain the performance differences observed between architectures in both our work and that of Kalogeropoulos et al. (2024). Overall, our findings underscore the potential of symmetry-aware metanetworks as a powerful approach for efficient and generalizable neural network optimization. Open-source code: https://github.com/daniuyter/scalegmn_amortization |
| title | Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2510.08300 |