Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.12601 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909905628168192 |
|---|---|
| author | Boufalis, Odysseas Carrasco-Pollo, Jorge Rosenthal, Joshua Terres-Caballero, Eduardo García-Castellanos, Alejandro |
| author_facet | Boufalis, Odysseas Carrasco-Pollo, Jorge Rosenthal, Joshua Terres-Caballero, Eduardo García-Castellanos, Alejandro |
| contents | Neural network parameterizations exhibit inherent symmetries that yield multiple equivalent minima within the loss landscape. Scale Graph Metanetworks (ScaleGMNs) explicitly leverage these symmetries by proposing an architecture equivariant to both permutation and parameter scaling transformations. Previous work by Ainsworth et al. (2023) addressed permutation symmetries through a computationally intensive combinatorial assignment problem, demonstrating that leveraging permutation symmetries alone can map networks into a shared loss basin. In this work, we extend their approach by also incorporating scaling symmetries, presenting an autoencoder framework utilizing ScaleGMNs as invariant encoders. Experimental results demonstrate that our method aligns Implicit Neural Representations (INRs) and Convolutional Neural Networks (CNNs) under both permutation and scaling symmetries without explicitly solving the assignment problem. This approach ensures that similar networks naturally converge within the same basin, facilitating model merging, i.e., smooth linear interpolation while avoiding regions of high loss. The code is publicly available on our GitHub repository. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_12601 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Symmetry-Aware Graph Metanetwork Autoencoders: Model Merging through Parameter Canonicalization Boufalis, Odysseas Carrasco-Pollo, Jorge Rosenthal, Joshua Terres-Caballero, Eduardo García-Castellanos, Alejandro Machine Learning Artificial Intelligence Neural network parameterizations exhibit inherent symmetries that yield multiple equivalent minima within the loss landscape. Scale Graph Metanetworks (ScaleGMNs) explicitly leverage these symmetries by proposing an architecture equivariant to both permutation and parameter scaling transformations. Previous work by Ainsworth et al. (2023) addressed permutation symmetries through a computationally intensive combinatorial assignment problem, demonstrating that leveraging permutation symmetries alone can map networks into a shared loss basin. In this work, we extend their approach by also incorporating scaling symmetries, presenting an autoencoder framework utilizing ScaleGMNs as invariant encoders. Experimental results demonstrate that our method aligns Implicit Neural Representations (INRs) and Convolutional Neural Networks (CNNs) under both permutation and scaling symmetries without explicitly solving the assignment problem. This approach ensures that similar networks naturally converge within the same basin, facilitating model merging, i.e., smooth linear interpolation while avoiding regions of high loss. The code is publicly available on our GitHub repository. |
| title | Symmetry-Aware Graph Metanetwork Autoencoders: Model Merging through Parameter Canonicalization |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2511.12601 |