Salvato in:
| Autori principali: | , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2512.11867 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912761325289472 |
|---|---|
| author | Zverev, Daniil Koepke, A. Sophia Henriques, Joao F. |
| author_facet | Zverev, Daniil Koepke, A. Sophia Henriques, Joao F. |
| contents | The use of synthetically generated data for training models is becoming a common practice. While generated data can augment the training data, repeated training on synthetic data raises concerns about distribution drift and degradation of performance due to contamination of the dataset. We investigate the consequences of this bootstrapping process through the lens of continual learning, drawing a connection to Generative Experience Replay (GER) methods. We present a statistical analysis showing that synthetic data introduces significant bias and variance into training objectives, weakening the reliability of maximum likelihood estimation. We provide empirical evidence showing that popular generative models collapse under repeated training with synthetic data. We quantify this degradation and show that state-of-the-art GER methods fail to maintain alignment in the latent space. Our findings raise critical concerns about the use of synthetic data in continual learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_11867 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | On the Dangers of Bootstrapping Generation for Continual Learning and Beyond Zverev, Daniil Koepke, A. Sophia Henriques, Joao F. Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Image and Video Processing The use of synthetically generated data for training models is becoming a common practice. While generated data can augment the training data, repeated training on synthetic data raises concerns about distribution drift and degradation of performance due to contamination of the dataset. We investigate the consequences of this bootstrapping process through the lens of continual learning, drawing a connection to Generative Experience Replay (GER) methods. We present a statistical analysis showing that synthetic data introduces significant bias and variance into training objectives, weakening the reliability of maximum likelihood estimation. We provide empirical evidence showing that popular generative models collapse under repeated training with synthetic data. We quantify this degradation and show that state-of-the-art GER methods fail to maintain alignment in the latent space. Our findings raise critical concerns about the use of synthetic data in continual learning. |
| title | On the Dangers of Bootstrapping Generation for Continual Learning and Beyond |
| topic | Machine Learning Artificial Intelligence Computer Vision and Pattern Recognition Image and Video Processing |
| url | https://arxiv.org/abs/2512.11867 |