Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2605.11546 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916003826368512 |
|---|---|
| author | Daniels, Sultan D'Ambrosia, Samuel H. DeWeese, Michael R. Sahai, Anant |
| author_facet | Daniels, Sultan D'Ambrosia, Samuel H. DeWeese, Michael R. Sahai, Anant |
| contents | Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_11546 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | The Entropy of Floating-Point Numbers Daniels, Sultan D'Ambrosia, Samuel H. DeWeese, Michael R. Sahai, Anant Information Theory Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a uniformly quantized random variable. Our approximation uncovers a different quantity that provides this link for floating-point quantization. Additionally, we prove that the entropy of a floating-point quantized random variable is approximately unchanged under scaling. Closed-form expressions for the floating-point entropy of common distributions are provided and compared to exact results. |
| title | The Entropy of Floating-Point Numbers |
| topic | Information Theory |
| url | https://arxiv.org/abs/2605.11546 |