Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.00199 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908800944963584 |
|---|---|
| author | Gegenfurtner, Johanna Marie Jacobsen, Albert Kjøller Borras, Naima Elosegui Mahou, Alejandro Valverde Arvanitidis, Georgios |
| author_facet | Gegenfurtner, Johanna Marie Jacobsen, Albert Kjøller Borras, Naima Elosegui Mahou, Alejandro Valverde Arvanitidis, Georgios |
| contents | Modern generative models can produce realistic samples, however, balancing memorisation and generalisation remains an open problem. We approach this challenge from a Bayesian perspective by focusing on the parameter space of flow matching and diffusion models and constructing a predictive posterior that better captures the variability of the data distribution. In particular, we capture the geometry of the loss using a Riemannian metric and leverage a flexible approximate posterior that adapts to the local structure of the loss landscape. This approach allows us to sample generative models that resemble the original model, but exhibit reduced memorisation. Empirically, we demonstrate that the proposed approach reduces memorisation while preserving generalisation. Further, we provide a theoretical analysis of our method, which explains our findings. Overall, our work illustrates how considering the geometry of the loss enables effective use of the parameter space, even for complex high-dimensional generative models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00199 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Reducing Memorisation in Generative Models via Riemannian Bayesian Inference Gegenfurtner, Johanna Marie Jacobsen, Albert Kjøller Borras, Naima Elosegui Mahou, Alejandro Valverde Arvanitidis, Georgios Machine Learning 53, 68 Modern generative models can produce realistic samples, however, balancing memorisation and generalisation remains an open problem. We approach this challenge from a Bayesian perspective by focusing on the parameter space of flow matching and diffusion models and constructing a predictive posterior that better captures the variability of the data distribution. In particular, we capture the geometry of the loss using a Riemannian metric and leverage a flexible approximate posterior that adapts to the local structure of the loss landscape. This approach allows us to sample generative models that resemble the original model, but exhibit reduced memorisation. Empirically, we demonstrate that the proposed approach reduces memorisation while preserving generalisation. Further, we provide a theoretical analysis of our method, which explains our findings. Overall, our work illustrates how considering the geometry of the loss enables effective use of the parameter space, even for complex high-dimensional generative models. |
| title | Reducing Memorisation in Generative Models via Riemannian Bayesian Inference |
| topic | Machine Learning 53, 68 |
| url | https://arxiv.org/abs/2602.00199 |