Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2504.08807 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866916686059274240 |
|---|---|
| author | Liu, HongZheng Tian, YiNuo Wu, Zhiyue |
| author_facet | Liu, HongZheng Tian, YiNuo Wu, Zhiyue |
| contents | This paper presents a unified framework, integrating information theory and statistical mechanics, to connect metric failure in high-dimensional data with emergence in complex systems. We propose the "Information Dilution Theorem," demonstrating that as dimensionality ($d$) increases, the mutual information efficiency between geometric metrics (e.g., Euclidean distance) and system states decays approximately as $O(1/d)$. This decay arises from the mismatch between linearly growing system entropy and sublinearly growing metric entropy, explaining the mechanism behind distance concentration. Building on this, we introduce information structural complexity ($C(S)$) based on the mutual information matrix spectrum and interaction encoding capacity ($C'$) derived from information bottleneck theory. The "Emergence Critical Theorem" states that when $C(S)$ exceeds $C'$, new global features inevitably emerge, satisfying a predefined mutual information threshold. This provides an operational criterion for self-organization and phase transitions. We discuss potential applications in physics, biology, and deep learning, suggesting potential directions like MI-based manifold learning (UMAP+) and offering a quantitative foundation for analyzing emergence across disciplines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_08807 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | The Exploratory Study on the Relationship Between the Failure of Distance Metrics in High-Dimensional Space and Emergent Phenomena Liu, HongZheng Tian, YiNuo Wu, Zhiyue Information Theory Statistical Mechanics Adaptation and Self-Organizing Systems This paper presents a unified framework, integrating information theory and statistical mechanics, to connect metric failure in high-dimensional data with emergence in complex systems. We propose the "Information Dilution Theorem," demonstrating that as dimensionality ($d$) increases, the mutual information efficiency between geometric metrics (e.g., Euclidean distance) and system states decays approximately as $O(1/d)$. This decay arises from the mismatch between linearly growing system entropy and sublinearly growing metric entropy, explaining the mechanism behind distance concentration. Building on this, we introduce information structural complexity ($C(S)$) based on the mutual information matrix spectrum and interaction encoding capacity ($C'$) derived from information bottleneck theory. The "Emergence Critical Theorem" states that when $C(S)$ exceeds $C'$, new global features inevitably emerge, satisfying a predefined mutual information threshold. This provides an operational criterion for self-organization and phase transitions. We discuss potential applications in physics, biology, and deep learning, suggesting potential directions like MI-based manifold learning (UMAP+) and offering a quantitative foundation for analyzing emergence across disciplines. |
| title | The Exploratory Study on the Relationship Between the Failure of Distance Metrics in High-Dimensional Space and Emergent Phenomena |
| topic | Information Theory Statistical Mechanics Adaptation and Self-Organizing Systems |
| url | https://arxiv.org/abs/2504.08807 |