Generator Matching: Generative modeling with arbitrary Markov processes
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916632459214848 |
|---|---|
| author | Holderrieth, Peter Havasi, Marton Yim, Jason Shaul, Neta Gat, Itai Jaakkola, Tommi Karrer, Brian Chen, Ricky T. Q. Lipman, Yaron |
| author_facet | Holderrieth, Peter Havasi, Marton Yim, Jason Shaul, Neta Gat, Itai Jaakkola, Tommi Karrer, Brian Chen, Ricky T. Q. Lipman, Yaron |
| contents | We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_20587 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Generator Matching: Generative modeling with arbitrary Markov processes Holderrieth, Peter Havasi, Marton Yim, Jason Shaul, Neta Gat, Itai Jaakkola, Tommi Karrer, Brian Chen, Ricky T. Q. Lipman, Yaron Machine Learning Artificial Intelligence We introduce Generator Matching, a modality-agnostic framework for generative modeling using arbitrary Markov processes. Generators characterize the infinitesimal evolution of a Markov process, which we leverage for generative modeling in a similar vein to flow matching: we construct conditional generators which generate single data points, then learn to approximate the marginal generator which generates the full data distribution. We show that Generator Matching unifies various generative modeling methods, including diffusion models, flow matching and discrete diffusion models. Furthermore, it expands the design space to new and unexplored Markov processes such as jump processes. Finally, Generator Matching enables the construction of superpositions of Markov generative models and enables the construction of multimodal models in a rigorous manner. We empirically validate our method on image and multimodal generation, e.g. showing that superposition with a jump process improves performance. |
| title | Generator Matching: Generative modeling with arbitrary Markov processes |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.20587 |