Normalized mutual information is a biased measure for classification and community detection
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918256425566208 |
|---|---|
| author | Jerdee, Maximilian Kirkley, Alec Newman, M. E. J. |
| author_facet | Jerdee, Maximilian Kirkley, Alec Newman, M. E. J. |
| contents | Normalized mutual information is widely used as a similarity measure for evaluating the performance of clustering and classification algorithms. In this paper, we argue that results returned by the normalized mutual information are biased for two reasons: first, because they ignore the information content of the contingency table and, second, because their symmetric normalization introduces spurious dependence on algorithm output. We introduce a modified version of the mutual information that remedies both of these shortcomings. As a practical demonstration of the importance of using an unbiased measure, we perform extensive numerical tests on a basket of popular algorithms for network community detection and show that one's conclusions about which algorithm is best are significantly affected by the biases in the traditional mutual information. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2307_01282 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Normalized mutual information is a biased measure for classification and community detection Jerdee, Maximilian Kirkley, Alec Newman, M. E. J. Social and Information Networks Machine Learning Normalized mutual information is widely used as a similarity measure for evaluating the performance of clustering and classification algorithms. In this paper, we argue that results returned by the normalized mutual information are biased for two reasons: first, because they ignore the information content of the contingency table and, second, because their symmetric normalization introduces spurious dependence on algorithm output. We introduce a modified version of the mutual information that remedies both of these shortcomings. As a practical demonstration of the importance of using an unbiased measure, we perform extensive numerical tests on a basket of popular algorithms for network community detection and show that one's conclusions about which algorithm is best are significantly affected by the biases in the traditional mutual information. |
| title | Normalized mutual information is a biased measure for classification and community detection |
| topic | Social and Information Networks Machine Learning |
| url | https://arxiv.org/abs/2307.01282 |