A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'
Fuente:
arXiv
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866929557890662400 |
|---|---|
| author | Borji, Ali |
| author_facet | Borji, Ali |
| contents | The study conducted by Shumailov et al. (2024) demonstrates that repeatedly training a generative model on synthetic data leads to model collapse. This finding has generated considerable interest and debate, particularly given that current models have nearly exhausted the available data. In this work, we investigate the effects of fitting a distribution (through Kernel Density Estimation, or KDE) or a model to the data, followed by repeated sampling from it. Our objective is to develop a theoretical understanding of the phenomenon observed by Shumailov et al. (2024). Our results indicate that the outcomes reported are a statistical phenomenon and may be unavoidable. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_12954 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data' Borji, Ali Machine Learning Artificial Intelligence The study conducted by Shumailov et al. (2024) demonstrates that repeatedly training a generative model on synthetic data leads to model collapse. This finding has generated considerable interest and debate, particularly given that current models have nearly exhausted the available data. In this work, we investigate the effects of fitting a distribution (through Kernel Density Estimation, or KDE) or a model to the data, followed by repeated sampling from it. Our objective is to develop a theoretical understanding of the phenomenon observed by Shumailov et al. (2024). Our results indicate that the outcomes reported are a statistical phenomenon and may be unavoidable. |
| title | A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data' |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.12954 |