Abstract Markov Random Fields
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910069454536704 |
|---|---|
| author | Lang, Leon de Mulatier, Clélia Quax, Rick Forré, Patrick |
| author_facet | Lang, Leon de Mulatier, Clélia Quax, Rick Forré, Patrick |
| contents | Markov random fields are known to be fully characterized by properties of their information diagrams, or I-diagrams. In particular, for Markov random fields, regions in the I-diagram corresponding to disconnected vertex sets in the graph vanish. Recently, I-diagrams have been generalized to F-diagrams, for a larger class of functions F satisfying the chain rule beyond Shannon entropy, such as Kullback-Leibler divergence and cross-entropy. In this work, we generalize the notion and characterization of Markov random fields to this larger class of functions F and investigate preliminary applications.
We define F-independences, F-mutual independences, and F-Markov random fields and characterize them by their F-diagram. In the process, we also define F-dual total correlation and prove that its vanishing is equivalent to F-mutual independence. We then apply our results to information functions F that are applied to probability mass functions. We show that if the probability distributions of a set of random variables are Markov random fields for the same graph, then we formally recover the notion of an F-Markov random field for that graph. We then study the Kullback-Leibler diagrams on specific Markov chains, leading to a visual representation of the second law of thermodynamics and a simple explicit derivation of the decomposition of the evidence lower bound for diffusion models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_02134 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Abstract Markov Random Fields Lang, Leon de Mulatier, Clélia Quax, Rick Forré, Patrick Information Theory Markov random fields are known to be fully characterized by properties of their information diagrams, or I-diagrams. In particular, for Markov random fields, regions in the I-diagram corresponding to disconnected vertex sets in the graph vanish. Recently, I-diagrams have been generalized to F-diagrams, for a larger class of functions F satisfying the chain rule beyond Shannon entropy, such as Kullback-Leibler divergence and cross-entropy. In this work, we generalize the notion and characterization of Markov random fields to this larger class of functions F and investigate preliminary applications. We define F-independences, F-mutual independences, and F-Markov random fields and characterize them by their F-diagram. In the process, we also define F-dual total correlation and prove that its vanishing is equivalent to F-mutual independence. We then apply our results to information functions F that are applied to probability mass functions. We show that if the probability distributions of a set of random variables are Markov random fields for the same graph, then we formally recover the notion of an F-Markov random field for that graph. We then study the Kullback-Leibler diagrams on specific Markov chains, leading to a visual representation of the second law of thermodynamics and a simple explicit derivation of the decomposition of the evidence lower bound for diffusion models. |
| title | Abstract Markov Random Fields |
| topic | Information Theory |
| url | https://arxiv.org/abs/2407.02134 |