Entropy estimators for Markovian sequences: A comparative analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909075432800256 |
|---|---|
| author | De Gregorio, Juan Sanchez, David Toral, Raul |
| author_facet | De Gregorio, Juan Sanchez, David Toral, Raul |
| contents | Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performance vary depending on the system being studied and the available data size. In this work we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_07547 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Entropy estimators for Markovian sequences: A comparative analysis De Gregorio, Juan Sanchez, David Toral, Raul Statistical Mechanics Chaotic Dynamics Data Analysis, Statistics and Probability Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performance vary depending on the system being studied and the available data size. In this work we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields. |
| title | Entropy estimators for Markovian sequences: A comparative analysis |
| topic | Statistical Mechanics Chaotic Dynamics Data Analysis, Statistics and Probability |
| url | https://arxiv.org/abs/2310.07547 |