Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2402.13536 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913238466166784 |
|---|---|
| author | Dotzel, Jordan Kotb, Bahaa Dotzel, James Abdelfattah, Mohamed Zhang, Zhiru |
| author_facet | Dotzel, Jordan Kotb, Bahaa Dotzel, James Abdelfattah, Mohamed Zhang, Zhiru |
| contents | Traditional methods, such as JPEG, perform image compression by operating on structural information, such as pixel values or frequency content. These methods are effective to bitrates around one bit per pixel (bpp) and higher at standard image sizes. In contrast, text-based semantic compression directly stores concepts and their relationships using natural language, which has evolved with humans to efficiently represent these salient concepts. These methods can operate at extremely low bitrates by disregarding structural information like location, size, and orientation. In this work, we use GPT-4V and DALL-E3 from OpenAI to explore the quality-compression frontier for image compression and identify the limitations of current technology. We push semantic compression as low as 100 $μ$bpp (up to $10,000\times$ smaller than JPEG) by introducing an iterative reflection process to improve the decoded image. We further hypothesize this 100 $μ$bpp level represents a soft limit on semantic compression at standard image resolutions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_13536 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Exploring the Limits of Semantic Image Compression at Micro-bits per Pixel Dotzel, Jordan Kotb, Bahaa Dotzel, James Abdelfattah, Mohamed Zhang, Zhiru Computer Vision and Pattern Recognition Artificial Intelligence Traditional methods, such as JPEG, perform image compression by operating on structural information, such as pixel values or frequency content. These methods are effective to bitrates around one bit per pixel (bpp) and higher at standard image sizes. In contrast, text-based semantic compression directly stores concepts and their relationships using natural language, which has evolved with humans to efficiently represent these salient concepts. These methods can operate at extremely low bitrates by disregarding structural information like location, size, and orientation. In this work, we use GPT-4V and DALL-E3 from OpenAI to explore the quality-compression frontier for image compression and identify the limitations of current technology. We push semantic compression as low as 100 $μ$bpp (up to $10,000\times$ smaller than JPEG) by introducing an iterative reflection process to improve the decoded image. We further hypothesize this 100 $μ$bpp level represents a soft limit on semantic compression at standard image resolutions. |
| title | Exploring the Limits of Semantic Image Compression at Micro-bits per Pixel |
| topic | Computer Vision and Pattern Recognition Artificial Intelligence |
| url | https://arxiv.org/abs/2402.13536 |