Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2501.15301 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929687757848576 |
|---|---|
| author | Xu, Xiangxiang Zheng, Lizhong |
| author_facet | Xu, Xiangxiang Zheng, Lizhong |
| contents | We study a separable design for computing information measures, where the information measure is computed from learned feature representations instead of raw data. Under mild assumptions on the feature representations, we demonstrate that a class of information measures admit such separable computation, including mutual information, $f$-information, Wyner's common information, G{á}cs--K{ö}rner common information, and Tishby's information bottleneck. Our development establishes several new connections between information measures and the statistical dependence structure. The characterizations also provide theoretical guarantees of practical designs for estimating information measures through representation learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_15301 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Separable Computation of Information Measures Xu, Xiangxiang Zheng, Lizhong Information Theory Machine Learning We study a separable design for computing information measures, where the information measure is computed from learned feature representations instead of raw data. Under mild assumptions on the feature representations, we demonstrate that a class of information measures admit such separable computation, including mutual information, $f$-information, Wyner's common information, G{á}cs--K{ö}rner common information, and Tishby's information bottleneck. Our development establishes several new connections between information measures and the statistical dependence structure. The characterizations also provide theoretical guarantees of practical designs for estimating information measures through representation learning. |
| title | Separable Computation of Information Measures |
| topic | Information Theory Machine Learning |
| url | https://arxiv.org/abs/2501.15301 |