Time-series Random Process Complexity Ranking Using a Bound on Conditional Differential Entropy
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915571767967744 |
|---|---|
| author | Ayers, Jacob Hahnloser, Richard Ulrich, Julia Krapp, Lothar Sebastian Nitschke, Remo Stoll, Sabine Bickel, Balthasar Furrer, Reinhard |
| author_facet | Ayers, Jacob Hahnloser, Richard Ulrich, Julia Krapp, Lothar Sebastian Nitschke, Remo Stoll, Sabine Bickel, Balthasar Furrer, Reinhard |
| contents | Conditional differential entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional processes from unknown distributions is often intractable. This paper builds on the information theoretic prediction error bounds established by Fang et al. \cite{fang2019generic}, which demonstrate that the conditional differential entropy \textbf{$h(X_k \mid X_{k-1},...,X_{k-m})$} is upper bounded by a function of the determinant of the covariance matrix of next-step prediction errors for any next step prediction model. We add to this theoretical framework by further increasing this bound by leveraging Hadamard's inequality and the positive semi-definite property of covariance matrices.
To see if these bounds can be used to rank the complexity of time series, we conducted two synthetic experiments: (1) controlled linear autoregressive processes with additive Gaussian noise, where we compare ordinary least squares prediction error entropy proxies to the true entropies of various additive noises, and (2) a complexity ranking task of bio-inspired synthetic audio data with unknown entropy, where neural network prediction errors are used to recover the known complexity ordering.
This framework provides a computationally tractable method for time-series complexity ranking using prediction errors from next-step prediction models, that maintains a theoretical foundation in information theory. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_20551 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Time-series Random Process Complexity Ranking Using a Bound on Conditional Differential Entropy Ayers, Jacob Hahnloser, Richard Ulrich, Julia Krapp, Lothar Sebastian Nitschke, Remo Stoll, Sabine Bickel, Balthasar Furrer, Reinhard Signal Processing Information Theory Audio and Speech Processing Methodology Machine Learning Conditional differential entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional processes from unknown distributions is often intractable. This paper builds on the information theoretic prediction error bounds established by Fang et al. \cite{fang2019generic}, which demonstrate that the conditional differential entropy \textbf{$h(X_k \mid X_{k-1},...,X_{k-m})$} is upper bounded by a function of the determinant of the covariance matrix of next-step prediction errors for any next step prediction model. We add to this theoretical framework by further increasing this bound by leveraging Hadamard's inequality and the positive semi-definite property of covariance matrices. To see if these bounds can be used to rank the complexity of time series, we conducted two synthetic experiments: (1) controlled linear autoregressive processes with additive Gaussian noise, where we compare ordinary least squares prediction error entropy proxies to the true entropies of various additive noises, and (2) a complexity ranking task of bio-inspired synthetic audio data with unknown entropy, where neural network prediction errors are used to recover the known complexity ordering. This framework provides a computationally tractable method for time-series complexity ranking using prediction errors from next-step prediction models, that maintains a theoretical foundation in information theory. |
| title | Time-series Random Process Complexity Ranking Using a Bound on Conditional Differential Entropy |
| topic | Signal Processing Information Theory Audio and Speech Processing Methodology Machine Learning |
| url | https://arxiv.org/abs/2510.20551 |