Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2504.19288 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913810174967808 |
|---|---|
| author | Shen, Yirong Gan, Lu Ling, Cong |
| author_facet | Shen, Yirong Gan, Lu Ling, Cong |
| contents | Relative Fisher information, also known as score matching, is a recently introduced learning method for parameter estimation. Fundamental relations between relative entropy and score matching have been established in the literature for scalar and isotropic Gaussian channels. This paper demonstrates that such relations hold for a much larger class of observation models. We introduce the vector channel where the perturbation is non-isotropic Gaussian noise. For such channels, we derive new representations that connect the $f$-divergence between two distributions to the estimation loss induced by mismatch at the decoder. This approach not only unifies but also greatly extends existing results from both the isotropic Gaussian and classical relative entropy frameworks. Building on this generalization, we extend De Bruijn's identity to mismatched non-isotropic Gaussian models and demonstrate that the connections to generative models naturally follow as a consequence application of this new result. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_19288 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Generalized Score Matching: Bridging $f$-Divergence and Statistical Estimation Under Correlated Noise Shen, Yirong Gan, Lu Ling, Cong Information Theory Relative Fisher information, also known as score matching, is a recently introduced learning method for parameter estimation. Fundamental relations between relative entropy and score matching have been established in the literature for scalar and isotropic Gaussian channels. This paper demonstrates that such relations hold for a much larger class of observation models. We introduce the vector channel where the perturbation is non-isotropic Gaussian noise. For such channels, we derive new representations that connect the $f$-divergence between two distributions to the estimation loss induced by mismatch at the decoder. This approach not only unifies but also greatly extends existing results from both the isotropic Gaussian and classical relative entropy frameworks. Building on this generalization, we extend De Bruijn's identity to mismatched non-isotropic Gaussian models and demonstrate that the connections to generative models naturally follow as a consequence application of this new result. |
| title | Generalized Score Matching: Bridging $f$-Divergence and Statistical Estimation Under Correlated Noise |
| topic | Information Theory |
| url | https://arxiv.org/abs/2504.19288 |