Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes
Fuente:
arXiv
Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866909403302592512 |
|---|---|
| author | Bodnarchuk, Iryna Mishura, Yuliya Ralchenko, Kostiantyn |
| author_facet | Bodnarchuk, Iryna Mishura, Yuliya Ralchenko, Kostiantyn |
| contents | We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, Rényi, generalized Rényi, Tsallis, Sharma-Mittal, and modified Shannon entropy, along with the Kullback-Leibler divergence. These measures are examined for several distributions, including gamma, chi-squared, exponential, Laplace, and log-normal distributions. We investigate the dependence of the entropy on the parameters of the respective distribution. We also study the convergence of Shannon entropy for certain probability distributions. Furthermore, we identify the extreme values of Shannon entropy for Gaussian vectors. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_15817 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes Bodnarchuk, Iryna Mishura, Yuliya Ralchenko, Kostiantyn Information Theory Probability 94A17, 60E05 We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, Rényi, generalized Rényi, Tsallis, Sharma-Mittal, and modified Shannon entropy, along with the Kullback-Leibler divergence. These measures are examined for several distributions, including gamma, chi-squared, exponential, Laplace, and log-normal distributions. We investigate the dependence of the entropy on the parameters of the respective distribution. We also study the convergence of Shannon entropy for certain probability distributions. Furthermore, we identify the extreme values of Shannon entropy for Gaussian vectors. |
| title | Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes |
| topic | Information Theory Probability 94A17, 60E05 |
| url | https://arxiv.org/abs/2411.15817 |