Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma
Fuente:
arXiv
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910000293609472 |
|---|---|
| author | Li, Cheuk Ting |
| author_facet | Li, Cheuk Ting |
| contents | We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the optimal one-to-one non-prefix code, and satisfies an elegant conditioning property. These properties make it useful for approximating the Shannon entropy in linear programming and maximum entropy problems, studying the optimal length of conditional encoding, and bounding the entropy of monotonic mixture distributions. In particular, it can give a bound $I(X;Y)+\log(I(X;Y)+3.4)+1$ for the strong functional representation lemma which is optimal within $2.8$ bits, and significantly improves upon the best known bound. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_13736 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma Li, Cheuk Ting Information Theory We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the optimal one-to-one non-prefix code, and satisfies an elegant conditioning property. These properties make it useful for approximating the Shannon entropy in linear programming and maximum entropy problems, studying the optimal length of conditional encoding, and bounding the entropy of monotonic mixture distributions. In particular, it can give a bound $I(X;Y)+\log(I(X;Y)+3.4)+1$ for the strong functional representation lemma which is optimal within $2.8$ bits, and significantly improves upon the best known bound. |
| title | Discrete Layered Entropy, Conditional Compression and a Tighter Strong Functional Representation Lemma |
| topic | Information Theory |
| url | https://arxiv.org/abs/2501.13736 |