CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910482682609664 |
|---|---|
| author | Meng, Zhiming Li, Hui Zhang, Zeyang Shen, Zhongwei Yu, Yunlong Song, Xiaoning Wu, Xiaojun |
| author_facet | Meng, Zhiming Li, Hui Zhang, Zeyang Shen, Zhongwei Yu, Yunlong Song, Xiaoning Wu, Xiaojun |
| contents | Generative models are widely utilized to model the distribution of fused images in the field of infrared and visible image fusion. However, current generative models based fusion methods often suffer from unstable training and slow inference speed. To tackle this problem, a novel fusion method based on consistency model is proposed, termed as CoMoFusion, which can generate the high-quality images and achieve fast image inference speed. In specific, the consistency model is used to construct multi-modal joint features in the latent space with the forward and reverse process. Then, the infrared and visible features extracted by the trained consistency model are fed into fusion module to generate the final fused image. In order to enhance the texture and salient information of fused images, a novel loss based on pixel value selection is also designed. Extensive experiments on public datasets illustrate that our method obtains the SOTA fusion performance compared with the existing fusion methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_20764 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model Meng, Zhiming Li, Hui Zhang, Zeyang Shen, Zhongwei Yu, Yunlong Song, Xiaoning Wu, Xiaojun Computer Vision and Pattern Recognition Generative models are widely utilized to model the distribution of fused images in the field of infrared and visible image fusion. However, current generative models based fusion methods often suffer from unstable training and slow inference speed. To tackle this problem, a novel fusion method based on consistency model is proposed, termed as CoMoFusion, which can generate the high-quality images and achieve fast image inference speed. In specific, the consistency model is used to construct multi-modal joint features in the latent space with the forward and reverse process. Then, the infrared and visible features extracted by the trained consistency model are fed into fusion module to generate the final fused image. In order to enhance the texture and salient information of fused images, a novel loss based on pixel value selection is also designed. Extensive experiments on public datasets illustrate that our method obtains the SOTA fusion performance compared with the existing fusion methods. |
| title | CoMoFusion: Fast and High-quality Fusion of Infrared and Visible Image with Consistency Model |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2405.20764 |