Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gerstgrasser, Matthias, Schaeffer, Rylan, Dey, Apratim, Rafailov, Rafael, Sleight, Henry, Hughes, John, Korbak, Tomasz, Agrawal, Rajashree, Pai, Dhruv, Gromov, Andrey, Roberts, Daniel A., Yang, Diyi, Donoho, David L., Koyejo, Sanmi
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning Artificial Intelligence Computation and Language Emerging Technologies
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
di: Kazdan, Joshua, et al.
Pubblicazione: (2024)

Universality of the $π^2/6$ Pathway in Avoiding Model Collapse
di: Dey, Apratim, et al.
Pubblicazione: (2024)

Position: Model Collapse Does Not Mean What You Think
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

In-Context Learning of Energy Functions
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Pretraining Scaling Laws for Generative Evaluations of Language Models
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Optimal Vector Compressed Sensing Using James Stein Shrinkage
di: Dey, Apratim, et al.
Pubblicazione: (2025)

Best-of-N Jailbreaking
di: Hughes, John, et al.
Pubblicazione: (2024)

Scale Dependent Data Duplication
di: Kazdan, Joshua, et al.
Pubblicazione: (2026)

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
di: Gupta, Isha, et al.
Pubblicazione: (2025)

Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
di: Wang, Tony T., et al.
Pubblicazione: (2024)

Bridging Associative Memory and Probabilistic Modeling
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
di: Lecomte, Victor, et al.
Pubblicazione: (2023)

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
di: Obbad, Elyas, et al.
Pubblicazione: (2024)

Efficient Prediction of Pass@k Scaling in Large Language Models
di: Kazdan, Joshua, et al.
Pubblicazione: (2025)

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
di: Miranda, Brando, et al.
Pubblicazione: (2023)

Evaluating the Robustness of Chinchilla Compute-Optimal Scaling
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Investigating Data Contamination for Pre-training Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2024)

Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
di: Denisov-Blanch, Yegor, et al.
Pubblicazione: (2026)

Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Scalable Ensembling For Mitigating Reward Overoptimisation
di: Ahmed, Ahmed M., et al.
Pubblicazione: (2024)

Aligning language models with human preferences
di: Korbak, Tomasz
Pubblicazione: (2024)

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms
di: Kazdan, Joshua, et al.
Pubblicazione: (2025)

How Do Large Language Monkeys Get Their Power (Laws)?
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Quantifying Variance in Evaluation Benchmarks
di: Madaan, Lovish, et al.
Pubblicazione: (2024)

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Causally Inspired Regularization Enables Domain General Representations
di: Salaudeen, Olawale, et al.
Pubblicazione: (2024)

Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes
di: Robertson, Zachary, et al.
Pubblicazione: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
di: Vo, Truong, et al.
Pubblicazione: (2025)

SWE-chat: Coding Agent Interactions From Real Users in the Wild
di: Baumann, Joachim, et al.
Pubblicazione: (2026)

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
di: Berglund, Lukas, et al.
Pubblicazione: (2023)

A Framework for Objective-Driven Dynamical Stochastic Fields
di: Zhang, Yibo Jacky, et al.
Pubblicazione: (2025)

Discovering Implicit Large Language Model Alignment Objectives
di: Chen, Edward, et al.
Pubblicazione: (2026)

High-Dimensional Markov-switching Ordinary Differential Processes
di: Tsai, Katherine, et al.
Pubblicazione: (2024)

Distributional Machine Unlearning via Selective Data Removal
di: Allouah, Youssef, et al.
Pubblicazione: (2025)

SCENEBench: An Audio Understanding Benchmark Grounded in Assistive and Industrial Use Cases
di: Iyer, Laya, et al.
Pubblicazione: (2026)

HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
di: Zhu, Junzhe, et al.
Pubblicazione: (2023)

Quantifying the Effect of Test Set Contamination on Generative Evaluations
di: Schaeffer, Rylan, et al.
Pubblicazione: (2026)

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models
di: Blevins, Terra, et al.
Pubblicazione: (2024)