Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2601.08882 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909989809946624 |
|---|---|
| author | Snyder, Thomas Yang, H. Lexie Schnake, Stefan Schotthöfer, Steffen |
| author_facet | Snyder, Thomas Yang, H. Lexie Schnake, Stefan Schotthöfer, Steffen |
| contents | Deploying geospatial foundation models on resource-constrained edge devices demands compact architectures that maintain high downstream performance. However, their large parameter counts and the accuracy loss often induced by compression limit practical adoption. In this work, we leverage manifold-constrained optimization framework DLRT to compress large vision transformer-based geospatial foundation models during transfer learning. By enforcing structured low-dimensional parameterizations aligned with downstream objectives, this approach achieves strong compression while preserving task-specific accuracy. We show that the method outperforms of-the-shelf low-rank methods as LoRA. Experiments on diverse geospatial benchmarks confirm substantial parameter reduction with minimal accuracy loss, enabling high-performing, on-device geospatial models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_08882 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Compressing Vision Transformers in Geospatial Transfer Learning with Manifold-Constrained Optimization Snyder, Thomas Yang, H. Lexie Schnake, Stefan Schotthöfer, Steffen Computer Vision and Pattern Recognition Artificial Intelligence Deploying geospatial foundation models on resource-constrained edge devices demands compact architectures that maintain high downstream performance. However, their large parameter counts and the accuracy loss often induced by compression limit practical adoption. In this work, we leverage manifold-constrained optimization framework DLRT to compress large vision transformer-based geospatial foundation models during transfer learning. By enforcing structured low-dimensional parameterizations aligned with downstream objectives, this approach achieves strong compression while preserving task-specific accuracy. We show that the method outperforms of-the-shelf low-rank methods as LoRA. Experiments on diverse geospatial benchmarks confirm substantial parameter reduction with minimal accuracy loss, enabling high-performing, on-device geospatial models. |
| title | Compressing Vision Transformers in Geospatial Transfer Learning with Manifold-Constrained Optimization |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2601.08882 |