Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909281062748160 |
|---|---|
| author | Wang, Xijun Ye, Dongshan Feng, Chenyuan Yang, Howard H. Chen, Xiang Quek, Tony Q. S. |
| author_facet | Wang, Xijun Ye, Dongshan Feng, Chenyuan Yang, Howard H. Chen, Xiang Quek, Tony Q. S. |
| contents | Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. However, existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. To address these limitations, we propose a novel trustworthy ISC framework. This approach leverages text extraction and segmentation mapping techniques to convert images into explainable semantics, while employing Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks. We also introduce a multi-rate ISC transmission protocol that dynamically adapts to both the received explainable semantic content and specific task requirements at the receiver. Simulation results demonstrate that our framework achieves explainable learning, decoupled training, and compatible transmission in various application scenarios. Finally, some intriguing research directions and application scenarios are identified. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_03806 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency Wang, Xijun Ye, Dongshan Feng, Chenyuan Yang, Howard H. Chen, Xiang Quek, Tony Q. S. Information Theory Machine Learning Networking and Internet Architecture Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. However, existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. To address these limitations, we propose a novel trustworthy ISC framework. This approach leverages text extraction and segmentation mapping techniques to convert images into explainable semantics, while employing Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks. We also introduce a multi-rate ISC transmission protocol that dynamically adapts to both the received explainable semantic content and specific task requirements at the receiver. Simulation results demonstrate that our framework achieves explainable learning, decoupled training, and compatible transmission in various application scenarios. Finally, some intriguing research directions and application scenarios are identified. |
| title | Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency |
| topic | Information Theory Machine Learning Networking and Internet Architecture |
| url | https://arxiv.org/abs/2408.03806 |