Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kazdan, Joshua, Schaeffer, Rylan, Dey, Apratim, Gerstgrasser, Matthias, Rafailov, Rafael, Donoho, David L., Koyejo, Sanmi
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
by: Gerstgrasser, Matthias, et al.
Published: (2024)

Position: Model Collapse Does Not Mean What You Think
by: Schaeffer, Rylan, et al.
Published: (2025)

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
by: Gupta, Isha, et al.
Published: (2025)

Scale Dependent Data Duplication
by: Kazdan, Joshua, et al.
Published: (2026)

Universality of the $π^2/6$ Pathway in Avoiding Model Collapse
by: Dey, Apratim, et al.
Published: (2024)

Efficient Prediction of Pass@k Scaling in Large Language Models
by: Kazdan, Joshua, et al.
Published: (2025)

Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
by: Denisov-Blanch, Yegor, et al.
Published: (2026)

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
by: Schaeffer, Rylan, et al.
Published: (2025)

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms
by: Kazdan, Joshua, et al.
Published: (2025)

In-Context Learning of Energy Functions
by: Schaeffer, Rylan, et al.
Published: (2024)

How Do Large Language Monkeys Get Their Power (Laws)?
by: Schaeffer, Rylan, et al.
Published: (2025)

Pretraining Scaling Laws for Generative Evaluations of Language Models
by: Schaeffer, Rylan, et al.
Published: (2025)

Optimal Vector Compressed Sensing Using James Stein Shrinkage
by: Dey, Apratim, et al.
Published: (2025)

Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models
by: Schaeffer, Rylan, et al.
Published: (2025)

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
by: Miranda, Brando, et al.
Published: (2023)

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
by: Obbad, Elyas, et al.
Published: (2024)

What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
by: Lecomte, Victor, et al.
Published: (2023)

Investigating Data Contamination for Pre-training Language Models
by: Jiang, Minhao, et al.
Published: (2024)

The Utility and Complexity of in- and out-of-Distribution Machine Unlearning
by: Allouah, Youssef, et al.
Published: (2024)

Quantifying Variance in Evaluation Benchmarks
by: Madaan, Lovish, et al.
Published: (2024)

Quantifying the Effect of Test Set Contamination on Generative Evaluations
by: Schaeffer, Rylan, et al.
Published: (2026)

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
by: Belakaria, Syrine, et al.
Published: (2025)

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
by: Schaeffer, Rylan, et al.
Published: (2024)

KGGen: Extracting Knowledge Graphs from Plain Text with Language Models
by: Mo, Belinda, et al.
Published: (2025)

Best-of-N Jailbreaking
by: Hughes, John, et al.
Published: (2024)

HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
by: Zhu, Junzhe, et al.
Published: (2023)

Scalable Ensembling For Mitigating Reward Overoptimisation
by: Ahmed, Ahmed M., et al.
Published: (2024)

SCENEBench: An Audio Understanding Benchmark Grounded in Assistive and Industrial Use Cases
by: Iyer, Laya, et al.
Published: (2026)

Are Domain Generalization Benchmarks with Accuracy on the Line Misspecified?
by: Salaudeen, Olawale, et al.
Published: (2025)

A Framework for Objective-Driven Dynamical Stochastic Fields
by: Zhang, Yibo Jacky, et al.
Published: (2025)

Evaluating the Robustness of Chinchilla Compute-Optimal Scaling
by: Schaeffer, Rylan, et al.
Published: (2025)

Optimization and Generalization Guarantees for Weight Normalization
by: Cisneros-Velarde, Pedro, et al.
Published: (2024)

The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behavior
by: Wang, Angelina, et al.
Published: (2025)

Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization
by: Chen, Zhi, et al.
Published: (2024)

Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
by: Gerstgrasser, Matthias, et al.
Published: (2023)

Stop Automating Peer Review Without Rigorous Evaluation
by: Baumann, Joachim, et al.
Published: (2026)

Causally Inspired Regularization Enables Domain General Representations
by: Salaudeen, Olawale, et al.
Published: (2024)

Why Do Safety Guardrails Degrade Across Languages?
by: Zhang, Max, et al.
Published: (2026)

Logits are All We Need to Adapt Closed Models
by: Hiranandani, Gaurush, et al.
Published: (2025)

Chain-of-Thought Hijacking
by: Zhao, Jianli, et al.
Published: (2025)