LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866911073771192320 |
|---|---|
| author | Wang, Tianyu Horiguchi, Akira Pang, Lingyou Priebe, Carey E. |
| author_facet | Wang, Tianyu Horiguchi, Akira Pang, Lingyou Priebe, Carey E. |
| contents | The increasing use of synthetic data from the public Internet has enhanced data usage efficiency in large language model (LLM) training. However, the potential threat of model collapse remains insufficiently explored. Existing studies primarily examine model collapse in a single model setting or rely solely on statistical surrogates. In this work, we introduce LLM Web Dynamics (LWD), an efficient framework for investigating model collapse at the network level. By simulating the Internet with a retrieval-augmented generation (RAG) database, we analyze the convergence pattern of model outputs. Furthermore, we provide theoretical guarantees for this convergence by drawing an analogy to interacting Gaussian Mixture Models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15690 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs Wang, Tianyu Horiguchi, Akira Pang, Lingyou Priebe, Carey E. Machine Learning Artificial Intelligence Social and Information Networks Methodology The increasing use of synthetic data from the public Internet has enhanced data usage efficiency in large language model (LLM) training. However, the potential threat of model collapse remains insufficiently explored. Existing studies primarily examine model collapse in a single model setting or rely solely on statistical surrogates. In this work, we introduce LLM Web Dynamics (LWD), an efficient framework for investigating model collapse at the network level. By simulating the Internet with a retrieval-augmented generation (RAG) database, we analyze the convergence pattern of model outputs. Furthermore, we provide theoretical guarantees for this convergence by drawing an analogy to interacting Gaussian Mixture Models. |
| title | LLM Web Dynamics: Tracing Model Collapse in a Network of LLMs |
| topic | Machine Learning Artificial Intelligence Social and Information Networks Methodology |
| url | https://arxiv.org/abs/2506.15690 |