On Transferring Transferability: Towards a Theory for Size Generalization
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915789411450880 |
|---|---|
| author | Levin, Eitan Ma, Yuxin Díaz, Mateo Villar, Soledad |
| author_facet | Levin, Eitan Ma, Yuxin Díaz, Mateo Villar, Soledad |
| contents | Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs are graphs, sets, and point clouds. Recent work on graph neural networks has explored whether a model trained on low-dimensional data can transfer its performance to higher-dimensional inputs. We extend this body of work by introducing a general framework for transferability across dimensions. We show that transferability corresponds precisely to continuity in a limit space formed by identifying small problem instances with equivalent large ones. This identification is driven by the data and the learning task. We instantiate our framework on existing architectures, and implement the necessary changes to ensure their transferability. Finally, we provide design principles for designing new transferable models. Numerical experiments support our findings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_23599 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On Transferring Transferability: Towards a Theory for Size Generalization Levin, Eitan Ma, Yuxin Díaz, Mateo Villar, Soledad Machine Learning Representation Theory Statistics Theory Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs are graphs, sets, and point clouds. Recent work on graph neural networks has explored whether a model trained on low-dimensional data can transfer its performance to higher-dimensional inputs. We extend this body of work by introducing a general framework for transferability across dimensions. We show that transferability corresponds precisely to continuity in a limit space formed by identifying small problem instances with equivalent large ones. This identification is driven by the data and the learning task. We instantiate our framework on existing architectures, and implement the necessary changes to ensure their transferability. Finally, we provide design principles for designing new transferable models. Numerical experiments support our findings. |
| title | On Transferring Transferability: Towards a Theory for Size Generalization |
| topic | Machine Learning Representation Theory Statistics Theory |
| url | https://arxiv.org/abs/2505.23599 |