A series of maximum entropy upper bounds of the differential entropy
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2016
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918269847339008 |
|---|---|
| author | Nielsen, Frank Nock, Richard |
| author_facet | Nielsen, Frank Nock, Richard |
| contents | We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bounds for upper bounding the differential entropy of Gaussian mixture models. This requires to calculate the raw moments and raw absolute moments of Gaussian mixtures in closed-form that may also be handy in statistical machine learning and information theory. We report on our experiments and discuss on the tightness of those bounds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1612_02954 |
| institution | arXiv |
| publishDate | 2016 |
| record_format | arxiv |
| spellingShingle | A series of maximum entropy upper bounds of the differential entropy Nielsen, Frank Nock, Richard Information Theory Computer Vision and Pattern Recognition Machine Learning We present a series of closed-form maximum entropy upper bounds for the differential entropy of a continuous univariate random variable and study the properties of that series. We then show how to use those generic bounds for upper bounding the differential entropy of Gaussian mixture models. This requires to calculate the raw moments and raw absolute moments of Gaussian mixtures in closed-form that may also be handy in statistical machine learning and information theory. We report on our experiments and discuss on the tightness of those bounds. |
| title | A series of maximum entropy upper bounds of the differential entropy |
| topic | Information Theory Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/1612.02954 |