A remark on conditional entropy
Fuente:
arXiv
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913295907160064 |
|---|---|
| author | Wang, Adam |
| author_facet | Wang, Adam |
| contents | The following note proves that conditional entropy of a sequence is almost time-reversal invariant, specifically they only differ by a small constant factor dependent only upon the forward and backward models that the entropies are being calculated with respect to. This gives rise to a numerical value that quantifies learnability, as well as a methodology to control for distributional shift between datasets. Rough guidelines are given for practitioners. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_02167 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A remark on conditional entropy Wang, Adam Information Theory The following note proves that conditional entropy of a sequence is almost time-reversal invariant, specifically they only differ by a small constant factor dependent only upon the forward and backward models that the entropies are being calculated with respect to. This gives rise to a numerical value that quantifies learnability, as well as a methodology to control for distributional shift between datasets. Rough guidelines are given for practitioners. |
| title | A remark on conditional entropy |
| topic | Information Theory |
| url | https://arxiv.org/abs/2404.02167 |