Language-Oriented Semantic Latent Representation for Image Transmission
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909205422669824 |
|---|---|
| author | Cicchetti, Giordano Grassucci, Eleonora Park, Jihong Choi, Jinho Barbarossa, Sergio Comminiello, Danilo |
| author_facet | Cicchetti, Giordano Grassucci, Eleonora Park, Jihong Choi, Jinho Barbarossa, Sergio Comminiello, Danilo |
| contents | In the new paradigm of semantic communication (SC), the focus is on delivering meanings behind bits by extracting semantic information from raw data. Recent advances in data-to-text models facilitate language-oriented SC, particularly for text-transformed image communication via image-to-text (I2T) encoding and text-to-image (T2I) decoding. However, although semantically aligned, the text is too coarse to precisely capture sophisticated visual features such as spatial locations, color, and texture, incurring a significant perceptual difference between intended and reconstructed images. To address this limitation, in this paper, we propose a novel language-oriented SC framework that communicates both text and a compressed image embedding and combines them using a latent diffusion model to reconstruct the intended image. Experimental results validate the potential of our approach, which transmits only 2.09\% of the original image size while achieving higher perceptual similarities in noisy communication channels compared to a baseline SC method that communicates only through text.The code is available at https://github.com/ispamm/Img2Img-SC/ . |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_09976 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Language-Oriented Semantic Latent Representation for Image Transmission Cicchetti, Giordano Grassucci, Eleonora Park, Jihong Choi, Jinho Barbarossa, Sergio Comminiello, Danilo Computer Vision and Pattern Recognition Signal Processing In the new paradigm of semantic communication (SC), the focus is on delivering meanings behind bits by extracting semantic information from raw data. Recent advances in data-to-text models facilitate language-oriented SC, particularly for text-transformed image communication via image-to-text (I2T) encoding and text-to-image (T2I) decoding. However, although semantically aligned, the text is too coarse to precisely capture sophisticated visual features such as spatial locations, color, and texture, incurring a significant perceptual difference between intended and reconstructed images. To address this limitation, in this paper, we propose a novel language-oriented SC framework that communicates both text and a compressed image embedding and combines them using a latent diffusion model to reconstruct the intended image. Experimental results validate the potential of our approach, which transmits only 2.09\% of the original image size while achieving higher perceptual similarities in noisy communication channels compared to a baseline SC method that communicates only through text.The code is available at https://github.com/ispamm/Img2Img-SC/ . |
| title | Language-Oriented Semantic Latent Representation for Image Transmission |
| topic | Computer Vision and Pattern Recognition Signal Processing |
| url | https://arxiv.org/abs/2405.09976 |