Image de couverture de livre

Escaping Collapse: The Strength of Weak Data for Large Language Model Training

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Amin, Kareem, Babakniya, Sara, Bie, Alex, Kong, Weiwei, Syed, Umar, Vassilvitskii, Sergei
Format:	Preprint
Publié:	2025
Sujets:	Machine Learning Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Clustering and Median Aggregation Improve Differentially Private Inference
par: Amin, Kareem, et autres
Publié: (2025)

Learning from Synthetic Data: Limitations of ERM
par: Amin, Kareem, et autres
Publié: (2026)

Private prediction for large-scale synthetic text generation
par: Amin, Kareem, et autres
Publié: (2024)

AI-rithmetic
par: Bie, Alex, et autres
Publié: (2026)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
par: Seddik, Mohamed El Amine, et autres
Publié: (2024)

Regurgitative Training: The Value of Real Data in Training Large Language Models
par: Zhang, Jinghui, et autres
Publié: (2024)

Reward Collapse in Aligning Large Language Models
par: Song, Ziang, et autres
Publié: (2023)

Can Large Language Models Integrate Spatial Data? Empirical Insights into Reasoning Strengths and Computational Weaknesses
par: Han, Bin, et autres
Publié: (2025)

EscapeBench: Towards Advancing Creative Intelligence of Language Model Agents
par: Qian, Cheng, et autres
Publié: (2024)

DIDS: Domain Impact-aware Data Sampling for Large Language Model Training
par: Shi, Weijie, et autres
Publié: (2025)

(Im)possibility of Automated Hallucination Detection in Large Language Models
par: Karbasi, Amin, et autres
Publié: (2025)

Generating Individual Travel Diaries Using Large Language Models Informed by Census and Land-Use Data
par: Amin, Sepehr Golrokh, et autres
Publié: (2025)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
par: Zhou, Zhanhui, et autres
Publié: (2024)

How to Synthesize Text Data without Model Collapse?
par: Zhu, Xuekai, et autres
Publié: (2024)

Training Optimal Large Diffusion Language Models
par: Ni, Jinjie, et autres
Publié: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
par: Zhang, Jun, et autres
Publié: (2025)

SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models
par: Yang, Yu, et autres
Publié: (2024)

Training Large Language Models to Predict Clinical Events
par: Turtel, Benjamin, et autres
Publié: (2026)

Scaling with Collapse: Efficient and Predictable Training of LLM Families
par: Bergsma, Shane, et autres
Publié: (2025)

Data Efficacy for Language Model Training
par: Dai, Yalun, et autres
Publié: (2025)

Lying Is Just a Phase: The Hidden Alignment Transition in Language Model Scaling
par: Amin, Adil
Publié: (2026)

Sample-aware Adaptive Structured Pruning for Large Language Models
par: Kong, Jun, et autres
Publié: (2025)

Detecting Training Data of Large Language Models via Expectation Maximization
par: Kim, Gyuwan, et autres
Publié: (2024)

Epistemic Diversity and Knowledge Collapse in Large Language Models
par: Wright, Dustin, et autres
Publié: (2025)

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models
par: Cheng, Jiale, et autres
Publié: (2024)

Self-Training Elicits Concise Reasoning in Large Language Models
par: Munkhbat, Tergel, et autres
Publié: (2025)

Sliding Window Attention Training for Efficient Large Language Models
par: Fu, Zichuan, et autres
Publié: (2025)

Learning Dynamics in Continual Pre-Training for Large Language Models
par: Wang, Xingjin, et autres
Publié: (2025)

Training Agents with Weakly Supervised Feedback from Large Language Models
par: Gong, Dihong, et autres
Publié: (2024)

Training and Evaluating Language Models with Template-based Data Generation
par: Zhang, Yifan
Publié: (2024)

Is Child-Directed Speech Effective Training Data for Language Models?
par: Feng, Steven Y., et autres
Publié: (2024)

L3Ms -- Lagrange Large Language Models
par: Dhillon, Guneet S., et autres
Publié: (2024)

LatentBreak: Jailbreaking Large Language Models through Latent Space Feedback
par: Mura, Raffaele, et autres
Publié: (2025)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
par: Chen, Mengzhao, et autres
Publié: (2024)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
par: Shi, Haizhou, et autres
Publié: (2024)

Query-Conditioned Test-Time Self-Training for Large Language Models
par: Song, Chaehee, et autres
Publié: (2026)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
par: Jia, Sheng, et autres
Publié: (2025)

SiLQ: Simple Large Language Model Quantization-Aware Training
par: Esser, Steven K., et autres
Publié: (2025)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
par: Wang, Yiping, et autres
Publié: (2025)

Interactions Across Blocks in Post-Training Quantization of Large Language Models
par: Shabanovi, Khasmamad, et autres
Publié: (2024)