Mutual Information Estimation via Normalizing Flows
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909210245070848 |
|---|---|
| author | Butakov, Ivan Tolmachev, Alexander Malanchuk, Sofia Neopryatnaya, Anna Frolov, Alexey |
| author_facet | Butakov, Ivan Tolmachev, Alexander Malanchuk, Sofia Neopryatnaya, Anna Frolov, Alexey |
| contents | We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_02187 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Mutual Information Estimation via Normalizing Flows Butakov, Ivan Tolmachev, Alexander Malanchuk, Sofia Neopryatnaya, Anna Frolov, Alexey Machine Learning Information Theory We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method. |
| title | Mutual Information Estimation via Normalizing Flows |
| topic | Machine Learning Information Theory |
| url | https://arxiv.org/abs/2403.02187 |