Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2405.04144 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915108619288576 |
|---|---|
| author | Wang, Yuhan Wu, Youlong Ma, Shuai Zhang, Ying-Jun Angela |
| author_facet | Wang, Yuhan Wu, Youlong Ma, Shuai Zhang, Ying-Jun Angela |
| contents | By extracting task-relevant information while maximally compressing the input, the information bottleneck (IB) principle has provided a guideline for learning effective and robust representations of the target inference. However, extending the idea to the multi-task learning scenario with joint consideration of generative tasks and traditional reconstruction tasks remains unexplored. This paper addresses this gap by reconsidering the lossy compression problem with diverse constraints on data reconstruction, perceptual quality, and classification accuracy. Firstly, we study two ternary relationships, namely, the rate-distortion-classification (RDC) and rate-perception-classification (RPC). For both RDC and RPC functions, we derive the closed-form expressions of the optimal rate for binary and Gaussian sources. These new results complement the IB principle and provide insights into effectively extracting task-oriented information to fulfill diverse objectives. Secondly, unlike prior research demonstrating a tradeoff between classification and perception in signal restoration problems, we prove that such a tradeoff does not exist in the RPC function and reveal that the source noise plays a decisive role in the classification-perception tradeoff. Finally, we implement a deep-learning-based image compression framework, incorporating multiple tasks related to distortion, perception, and classification. The experimental results coincide with the theoretical analysis and verify the effectiveness of our generalized IB in balancing various task objectives. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_04144 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Task-Oriented Lossy Compression with Data, Perception, and Classification Constraints Wang, Yuhan Wu, Youlong Ma, Shuai Zhang, Ying-Jun Angela Information Theory By extracting task-relevant information while maximally compressing the input, the information bottleneck (IB) principle has provided a guideline for learning effective and robust representations of the target inference. However, extending the idea to the multi-task learning scenario with joint consideration of generative tasks and traditional reconstruction tasks remains unexplored. This paper addresses this gap by reconsidering the lossy compression problem with diverse constraints on data reconstruction, perceptual quality, and classification accuracy. Firstly, we study two ternary relationships, namely, the rate-distortion-classification (RDC) and rate-perception-classification (RPC). For both RDC and RPC functions, we derive the closed-form expressions of the optimal rate for binary and Gaussian sources. These new results complement the IB principle and provide insights into effectively extracting task-oriented information to fulfill diverse objectives. Secondly, unlike prior research demonstrating a tradeoff between classification and perception in signal restoration problems, we prove that such a tradeoff does not exist in the RPC function and reveal that the source noise plays a decisive role in the classification-perception tradeoff. Finally, we implement a deep-learning-based image compression framework, incorporating multiple tasks related to distortion, perception, and classification. The experimental results coincide with the theoretical analysis and verify the effectiveness of our generalized IB in balancing various task objectives. |
| title | Task-Oriented Lossy Compression with Data, Perception, and Classification Constraints |
| topic | Information Theory |
| url | https://arxiv.org/abs/2405.04144 |