Saved in:
| Main Authors: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.21841 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908288188153856 |
|---|---|
| author | Li, Jingtao Liu, Yingyi Wang, Xinyu Peng, Yunning Sun, Chen Wang, Shaoyu Sun, Zhendong Ke, Tian Jiang, Xiao Lu, Tangwei Zhao, Anran Zhong, Yanfei |
| author_facet | Li, Jingtao Liu, Yingyi Wang, Xinyu Peng, Yunning Sun, Chen Wang, Shaoyu Sun, Zhendong Ke, Tian Jiang, Xiao Lu, Tangwei Zhao, Anran Zhong, Yanfei |
| contents | Advanced interpretation of hyperspectral remote sensing images benefits many precise Earth observation tasks. Recently, visual foundation models have promoted the remote sensing interpretation but concentrating on RGB and multispectral images. Due to the varied hyperspectral channels,existing foundation models would face image-by-image tuning situation, imposing great pressure on hardware and time resources. In this paper, we propose a tuning-free hyperspectral foundation model called HyperFree, by adapting the existing visual prompt engineering. To process varied channel numbers, we design a learned weight dictionary covering full-spectrum from $0.4 \sim 2.5 \, μ\text{m}$, supporting to build the embedding layer dynamically. To make the prompt design more tractable, HyperFree can generate multiple semantic-aware masks for one prompt by treating feature distance as semantic-similarity. After pre-training HyperFree on constructed large-scale high-resolution hyperspectral images, HyperFree (1 prompt) has shown comparable results with specialized models (5 shots) on 5 tasks and 11 datasets.Code and dataset are accessible at https://rsidea.whu.edu.cn/hyperfree.htm. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_21841 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery Li, Jingtao Liu, Yingyi Wang, Xinyu Peng, Yunning Sun, Chen Wang, Shaoyu Sun, Zhendong Ke, Tian Jiang, Xiao Lu, Tangwei Zhao, Anran Zhong, Yanfei Computer Vision and Pattern Recognition Advanced interpretation of hyperspectral remote sensing images benefits many precise Earth observation tasks. Recently, visual foundation models have promoted the remote sensing interpretation but concentrating on RGB and multispectral images. Due to the varied hyperspectral channels,existing foundation models would face image-by-image tuning situation, imposing great pressure on hardware and time resources. In this paper, we propose a tuning-free hyperspectral foundation model called HyperFree, by adapting the existing visual prompt engineering. To process varied channel numbers, we design a learned weight dictionary covering full-spectrum from $0.4 \sim 2.5 \, μ\text{m}$, supporting to build the embedding layer dynamically. To make the prompt design more tractable, HyperFree can generate multiple semantic-aware masks for one prompt by treating feature distance as semantic-similarity. After pre-training HyperFree on constructed large-scale high-resolution hyperspectral images, HyperFree (1 prompt) has shown comparable results with specialized models (5 shots) on 5 tasks and 11 datasets.Code and dataset are accessible at https://rsidea.whu.edu.cn/hyperfree.htm. |
| title | HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.21841 |