Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866929419858214912 |
|---|---|
| author | Li, Han Li, Shaohui Ding, Shuangrui Dai, Wenrui Cao, Maida Li, Chenglin Zou, Junni Xiong, Hongkai |
| author_facet | Li, Han Li, Shaohui Ding, Shuangrui Dai, Wenrui Cao, Maida Li, Chenglin Zou, Junni Xiong, Hongkai |
| contents | Image compression for machine and human vision (ICMH) has gained increasing attention in recent years. Existing ICMH methods are limited by high training and storage overheads due to heavy design of task-specific networks. To address this issue, in this paper, we develop a novel lightweight adapter-based tuning framework for ICMH, named Adapt-ICMH, that better balances task performance and bitrates with reduced overheads. We propose a spatial-frequency modulation adapter (SFMA) that simultaneously eliminates non-semantic redundancy with a spatial modulation adapter, and enhances task-relevant frequency components and suppresses task-irrelevant frequency components with a frequency modulation adapter. The proposed adapter is plug-and-play and compatible with almost all existing learned image compression models without compromising the performance of pre-trained models. Experiments demonstrate that Adapt-ICMH consistently outperforms existing ICMH frameworks on various machine vision tasks with fewer fine-tuned parameters and reduced computational complexity. Code will be released at https://github.com/qingshi9974/ECCV2024-AdpatICMH . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_09853 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation Li, Han Li, Shaohui Ding, Shuangrui Dai, Wenrui Cao, Maida Li, Chenglin Zou, Junni Xiong, Hongkai Computer Vision and Pattern Recognition Image compression for machine and human vision (ICMH) has gained increasing attention in recent years. Existing ICMH methods are limited by high training and storage overheads due to heavy design of task-specific networks. To address this issue, in this paper, we develop a novel lightweight adapter-based tuning framework for ICMH, named Adapt-ICMH, that better balances task performance and bitrates with reduced overheads. We propose a spatial-frequency modulation adapter (SFMA) that simultaneously eliminates non-semantic redundancy with a spatial modulation adapter, and enhances task-relevant frequency components and suppresses task-irrelevant frequency components with a frequency modulation adapter. The proposed adapter is plug-and-play and compatible with almost all existing learned image compression models without compromising the performance of pre-trained models. Experiments demonstrate that Adapt-ICMH consistently outperforms existing ICMH frameworks on various machine vision tasks with fewer fine-tuned parameters and reduced computational complexity. Code will be released at https://github.com/qingshi9974/ECCV2024-AdpatICMH . |
| title | Image Compression for Machine and Human Vision with Spatial-Frequency Adaptation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.09853 |