Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914263845568512 |
|---|---|
| author | Dębowski, Łukasz |
| author_facet | Dębowski, Łukasz |
| contents | We construct multiperiodic processes -- a simple example of stationary ergodic (but not mixing) processes over natural numbers that enjoy the vanishing entropy rate under a mild condition. Multiperiodic processes are supported on randomly shifted deterministic sequences called multiperiodic sequences, which can be efficiently generated using an algorithm called the Infinite Clock. Under a suitable parameterization, multiperiodic sequences exhibit relative frequencies of particular numbers given by Zipf's law. Exactly in the same setting, the respective multiperiodic processes satisfy an asymptotic power-law growth of block entropy, called Hilberg's law. Hilberg's law is deemed to hold for statistical language models, in particular. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_09049 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy Dębowski, Łukasz Information Theory Machine Learning Statistics Theory 60G10 (Primary) 62M20, 94A17 (Secondary) We construct multiperiodic processes -- a simple example of stationary ergodic (but not mixing) processes over natural numbers that enjoy the vanishing entropy rate under a mild condition. Multiperiodic processes are supported on randomly shifted deterministic sequences called multiperiodic sequences, which can be efficiently generated using an algorithm called the Infinite Clock. Under a suitable parameterization, multiperiodic sequences exhibit relative frequencies of particular numbers given by Zipf's law. Exactly in the same setting, the respective multiperiodic processes satisfy an asymptotic power-law growth of block entropy, called Hilberg's law. Hilberg's law is deemed to hold for statistical language models, in particular. |
| title | Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy |
| topic | Information Theory Machine Learning Statistics Theory 60G10 (Primary) 62M20, 94A17 (Secondary) |
| url | https://arxiv.org/abs/2302.09049 |