Estimating Rate-Distortion Functions Using the Energy-Based Model
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918104137728000 |
|---|---|
| author | Wu, Shitong Xu, Sicheng Chen, Lingyi Wu, Huihui Zhang, Wenyi |
| author_facet | Wu, Shitong Xu, Sicheng Chen, Lingyi Wu, Huihui Zhang, Wenyi |
| contents | The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The Blahut-Arimoto (BA) algorithm, as a classical algorithm to compute RD functions, encounters computational challenges when applied to high-dimensional scenarios. In recent years, many neural methods have attempted to compute high-dimensional RD problems from the perspective of implicit generative models. Nevertheless, these approaches often neglect the reconstruction of the optimal conditional distribution or rely on unreasonable prior assumptions. In face of these issues, we propose an innovative energy-based modeling framework that leverages the connection between the RD dual form and the free energy in statistical physics, achieving effective reconstruction of the optimal conditional distribution.The proposed algorithm requires training only a single neural network and circumvents the challenge of computing the normalization factor in energy-based models using the Markov chain Monte Carlo (MCMC) sampling. Experimental results demonstrate the significant effectiveness of the proposed algorithm in estimating high-dimensional RD functions and reconstructing the optimal conditional distribution. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15700 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Estimating Rate-Distortion Functions Using the Energy-Based Model Wu, Shitong Xu, Sicheng Chen, Lingyi Wu, Huihui Zhang, Wenyi Information Theory The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The Blahut-Arimoto (BA) algorithm, as a classical algorithm to compute RD functions, encounters computational challenges when applied to high-dimensional scenarios. In recent years, many neural methods have attempted to compute high-dimensional RD problems from the perspective of implicit generative models. Nevertheless, these approaches often neglect the reconstruction of the optimal conditional distribution or rely on unreasonable prior assumptions. In face of these issues, we propose an innovative energy-based modeling framework that leverages the connection between the RD dual form and the free energy in statistical physics, achieving effective reconstruction of the optimal conditional distribution.The proposed algorithm requires training only a single neural network and circumvents the challenge of computing the normalization factor in energy-based models using the Markov chain Monte Carlo (MCMC) sampling. Experimental results demonstrate the significant effectiveness of the proposed algorithm in estimating high-dimensional RD functions and reconstructing the optimal conditional distribution. |
| title | Estimating Rate-Distortion Functions Using the Energy-Based Model |
| topic | Information Theory |
| url | https://arxiv.org/abs/2507.15700 |