Properties of the Strong Data Processing Constant for Rényi Divergence
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917666321596416 |
|---|---|
| author | Jin, Lifu Esposito, Amedeo Roberto Gastpar, Michael |
| author_facet | Jin, Lifu Esposito, Amedeo Roberto Gastpar, Michael |
| contents | Strong data processing inequalities (SDPI) are an important object of study in Information Theory and have been well studied for $f$-divergences. Universal upper and lower bounds have been provided along with several applications, connecting them to impossibility (converse) results, concentration of measure, hypercontractivity, and so on. In this paper, we study Rényi divergence and the corresponding SDPI constant whose behavior seems to deviate from that of ordinary $Φ$-divergences. In particular, one can find examples showing that the universal upper bound relating its SDPI constant to the one of Total Variation does not hold in general. In this work, we prove, however, that the universal lower bound involving the SDPI constant of the Chi-square divergence does indeed hold. Furthermore, we also provide a characterization of the distribution that achieves the supremum when $α$ is equal to $2$ and consequently compute the SDPI constant for Rényi divergence of the general binary channel. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10656 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Properties of the Strong Data Processing Constant for Rényi Divergence Jin, Lifu Esposito, Amedeo Roberto Gastpar, Michael Information Theory Strong data processing inequalities (SDPI) are an important object of study in Information Theory and have been well studied for $f$-divergences. Universal upper and lower bounds have been provided along with several applications, connecting them to impossibility (converse) results, concentration of measure, hypercontractivity, and so on. In this paper, we study Rényi divergence and the corresponding SDPI constant whose behavior seems to deviate from that of ordinary $Φ$-divergences. In particular, one can find examples showing that the universal upper bound relating its SDPI constant to the one of Total Variation does not hold in general. In this work, we prove, however, that the universal lower bound involving the SDPI constant of the Chi-square divergence does indeed hold. Furthermore, we also provide a characterization of the distribution that achieves the supremum when $α$ is equal to $2$ and consequently compute the SDPI constant for Rényi divergence of the general binary channel. |
| title | Properties of the Strong Data Processing Constant for Rényi Divergence |
| topic | Information Theory |
| url | https://arxiv.org/abs/2403.10656 |