Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2404.07311 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909486713667584 |
|---|---|
| author | Joudeh, Basheer Škorić, Boris |
| author_facet | Joudeh, Basheer Škorić, Boris |
| contents | We calculate the average differential entropy of a $q$-component Gaussian mixture in $\mathbb R^n$. For simplicity, all components have covariance matrix $σ^2 {\mathbf 1}$, while the means $\{\mathbf{W}_i\}_{i=1}^{q}$ are i.i.d. Gaussian vectors with zero mean and covariance $s^2 {\mathbf 1}$. We obtain a series expansion in $μ=s^2/σ^2$ for the average differential entropy up to order $\mathcal{O}(μ^2)$, and we provide a recipe to calculate higher order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2404_07311 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Average entropy of Gaussian mixtures Joudeh, Basheer Škorić, Boris Information Theory We calculate the average differential entropy of a $q$-component Gaussian mixture in $\mathbb R^n$. For simplicity, all components have covariance matrix $σ^2 {\mathbf 1}$, while the means $\{\mathbf{W}_i\}_{i=1}^{q}$ are i.i.d. Gaussian vectors with zero mean and covariance $s^2 {\mathbf 1}$. We obtain a series expansion in $μ=s^2/σ^2$ for the average differential entropy up to order $\mathcal{O}(μ^2)$, and we provide a recipe to calculate higher order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature. |
| title | Average entropy of Gaussian mixtures |
| topic | Information Theory |
| url | https://arxiv.org/abs/2404.07311 |