Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918239451217920 |
|---|---|
| author | Semenenko, Alexander Butakov, Ivan Frolov, Alexey Oseledets, Ivan |
| author_facet | Semenenko, Alexander Butakov, Ivan Frolov, Alexey Oseledets, Ivan |
| contents | Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and nullification only under statistical independence, we demonstrate that SMI is highly susceptible to data manipulation and exhibits counterintuitive behavior. Through extensive benchmarking and theoretical analysis, we show that SMI saturates easily, fails to detect increases in statistical dependence, prioritizes redundancy over informative content, and in some cases, performs worse than correlation coefficient. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_04053 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence Semenenko, Alexander Butakov, Ivan Frolov, Alexey Oseledets, Ivan Machine Learning Information Theory 94A16, 68T07, 94A17 E.4; H.1.1 Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and nullification only under statistical independence, we demonstrate that SMI is highly susceptible to data manipulation and exhibits counterintuitive behavior. Through extensive benchmarking and theoretical analysis, we show that SMI saturates easily, fails to detect increases in statistical dependence, prioritizes redundancy over informative content, and in some cases, performs worse than correlation coefficient. |
| title | Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence |
| topic | Machine Learning Information Theory 94A16, 68T07, 94A17 E.4; H.1.1 |
| url | https://arxiv.org/abs/2506.04053 |