Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2503.10832 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915197358178304 |
|---|---|
| author | Malidarreh, Parisa Boodaghi Saurav, Jillur Rahman Pham, Thuong Le Hoai Hajighasemi, Amir Samadi, Anahita Maydeo, Saurabh Shrinivas Nasr, Mohammad Sadegh Luber, Jacob M. |
| author_facet | Malidarreh, Parisa Boodaghi Saurav, Jillur Rahman Pham, Thuong Le Hoai Hajighasemi, Amir Samadi, Anahita Maydeo, Saurabh Shrinivas Nasr, Mohammad Sadegh Luber, Jacob M. |
| contents | Vector Quantization (VQ) techniques face significant challenges in codebook utilization, limiting reconstruction fidelity in image modeling. We introduce a Dual Codebook mechanism that effectively addresses this limitation by partitioning the representation into complementary global and local components. The global codebook employs a lightweight transformer for concurrent updates of all code vectors, while the local codebook maintains precise feature representation through deterministic selection. This complementary approach is trained from scratch without requiring pre-trained knowledge. Experimental evaluation across multiple standard benchmark datasets demonstrates state-of-the-art reconstruction quality while using a compact codebook of size 512 - half the size of previous methods that require pre-training. Our approach achieves significant FID improvements across diverse image domains, particularly excelling in scene and face reconstruction tasks. These results establish Dual Codebook VQ as an efficient paradigm for high-fidelity image reconstruction with significantly reduced computational requirements. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_10832 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Dual Codebook VQ: Enhanced Image Reconstruction with Reduced Codebook Size Malidarreh, Parisa Boodaghi Saurav, Jillur Rahman Pham, Thuong Le Hoai Hajighasemi, Amir Samadi, Anahita Maydeo, Saurabh Shrinivas Nasr, Mohammad Sadegh Luber, Jacob M. Computer Vision and Pattern Recognition Vector Quantization (VQ) techniques face significant challenges in codebook utilization, limiting reconstruction fidelity in image modeling. We introduce a Dual Codebook mechanism that effectively addresses this limitation by partitioning the representation into complementary global and local components. The global codebook employs a lightweight transformer for concurrent updates of all code vectors, while the local codebook maintains precise feature representation through deterministic selection. This complementary approach is trained from scratch without requiring pre-trained knowledge. Experimental evaluation across multiple standard benchmark datasets demonstrates state-of-the-art reconstruction quality while using a compact codebook of size 512 - half the size of previous methods that require pre-training. Our approach achieves significant FID improvements across diverse image domains, particularly excelling in scene and face reconstruction tasks. These results establish Dual Codebook VQ as an efficient paradigm for high-fidelity image reconstruction with significantly reduced computational requirements. |
| title | Dual Codebook VQ: Enhanced Image Reconstruction with Reduced Codebook Size |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.10832 |