Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910985179103232 |
|---|---|
| author | Tseng, Gabriel Fuller, Anthony Reil, Marlena Herzog, Henry Beukema, Patrick Bastani, Favyen Green, James R. Shelhamer, Evan Kerner, Hannah Rolnick, David |
| author_facet | Tseng, Gabriel Fuller, Anthony Reil, Marlena Herzog, Henry Beukema, Patrick Bastani, Favyen Green, James R. Shelhamer, Evan Kerner, Hannah Rolnick, David |
| contents | We introduce a highly multimodal transformer to represent many remote sensing modalities - multispectral optical, synthetic aperture radar, elevation, weather, pseudo-labels, and more - across space and time. These inputs are useful for diverse remote sensing tasks, such as crop mapping and flood detection. However, learning shared representations of remote sensing data is challenging, given the diversity of relevant data modalities, and because objects of interest vary massively in scale, from small boats (1-2 pixels and fast) to glaciers (thousands of pixels and slow). We present a novel self-supervised learning algorithm that extracts multi-scale features across a flexible set of input modalities through masked modeling. Our dual global and local contrastive losses differ in their targets (deep representations vs. shallow input projections) and masking strategies (structured vs. not). Our Galileo is a single generalist model that outperforms SoTA specialist models for satellite images and pixel time series across eleven benchmarks and multiple tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_09356 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Galileo: Learning Global & Local Features of Many Remote Sensing Modalities Tseng, Gabriel Fuller, Anthony Reil, Marlena Herzog, Henry Beukema, Patrick Bastani, Favyen Green, James R. Shelhamer, Evan Kerner, Hannah Rolnick, David Computer Vision and Pattern Recognition We introduce a highly multimodal transformer to represent many remote sensing modalities - multispectral optical, synthetic aperture radar, elevation, weather, pseudo-labels, and more - across space and time. These inputs are useful for diverse remote sensing tasks, such as crop mapping and flood detection. However, learning shared representations of remote sensing data is challenging, given the diversity of relevant data modalities, and because objects of interest vary massively in scale, from small boats (1-2 pixels and fast) to glaciers (thousands of pixels and slow). We present a novel self-supervised learning algorithm that extracts multi-scale features across a flexible set of input modalities through masked modeling. Our dual global and local contrastive losses differ in their targets (deep representations vs. shallow input projections) and masking strategies (structured vs. not). Our Galileo is a single generalist model that outperforms SoTA specialist models for satellite images and pixel time series across eleven benchmarks and multiple tasks. |
| title | Galileo: Learning Global & Local Features of Many Remote Sensing Modalities |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.09356 |