Weight Space Representation Learning via Neural Field Adaptation
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910039851139072 |
|---|---|
| author | Yang, Zhuoqian Salzmann, Mathieu Süsstrunk, Sabine |
| author_facet | Yang, Zhuoqian Salzmann, Mathieu Süsstrunk, Sabine |
| contents | In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields. Our key insight is that constraining the optimization space through a pre-trained base model and low-rank adaptation (LoRA) can induce structure in weight space. Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that multiplicative LoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure. When used with latent diffusion models, multiplicative LoRA weights enable higher-quality generation than existing weight-space methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_01759 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Weight Space Representation Learning via Neural Field Adaptation Yang, Zhuoqian Salzmann, Mathieu Süsstrunk, Sabine Machine Learning Artificial Intelligence In this work, we investigate the potential of weights to serve as effective representations, focusing on neural fields. Our key insight is that constraining the optimization space through a pre-trained base model and low-rank adaptation (LoRA) can induce structure in weight space. Across reconstruction, generation, and analysis tasks on 2D and 3D data, we find that multiplicative LoRA weights achieve high representation quality while exhibiting distinctiveness and semantic structure. When used with latent diffusion models, multiplicative LoRA weights enable higher-quality generation than existing weight-space methods. |
| title | Weight Space Representation Learning via Neural Field Adaptation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.01759 |