Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2604.10347 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910122081517568 |
|---|---|
| author | Kage, Patrick Andreadis, Pavlos |
| author_facet | Kage, Patrick Andreadis, Pavlos |
| contents | Vision foundation models have been shown to be effective at processing satellite imagery into representations fit for downstream tasks, however, creating models which operate over multiple spatial resolutions and modes is challenging. This paper presents Scale-ALiBi, a linear bias transformer attention mechanism with a spatial encoding bias to relationships between image patches at different ground sample distance scales. We provide an implementation of Scale-ALiBi over a dataset of aligned high- and low-resolution optical and low-resolution SAR satellite imagery data using a triple-contrastive and reconstructive architecture, show an improvement on the GEO-Bench benchmark, and release the newly curated dataset publicly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_10347 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Multi-modal, multi-scale representation learning for satellite imagery analysis just needs a good ALiBi Kage, Patrick Andreadis, Pavlos Computer Vision and Pattern Recognition Vision foundation models have been shown to be effective at processing satellite imagery into representations fit for downstream tasks, however, creating models which operate over multiple spatial resolutions and modes is challenging. This paper presents Scale-ALiBi, a linear bias transformer attention mechanism with a spatial encoding bias to relationships between image patches at different ground sample distance scales. We provide an implementation of Scale-ALiBi over a dataset of aligned high- and low-resolution optical and low-resolution SAR satellite imagery data using a triple-contrastive and reconstructive architecture, show an improvement on the GEO-Bench benchmark, and release the newly curated dataset publicly. |
| title | Multi-modal, multi-scale representation learning for satellite imagery analysis just needs a good ALiBi |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.10347 |