Copertina

Does Training on Synthetic Data Make Models Less Robust?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Lingze, Pavlick, Ellie
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

How Do Vision-Language Models Process Conflicting Information Across Modalities?
di: Hua, Tianze, et al.
Pubblicazione: (2025)

How Do Language Models Compose Functions?
di: Khandelwal, Apoorv, et al.
Pubblicazione: (2025)

Talking Heads: Understanding Inter-layer Communication in Transformer Language Models
di: Merullo, Jack, et al.
Pubblicazione: (2024)

What is an "Abstract Reasoner"? Revisiting Experiments and Arguments about Large Language Models
di: Yun, Tian, et al.
Pubblicazione: (2025)

mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models?
di: Hua, Tianze, et al.
Pubblicazione: (2024)

Shared Lexical Task Representations Explain Behavioral Variability In LLMs
di: Yang, Zhuonan, et al.
Pubblicazione: (2026)

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
di: Seddik, Mohamed El Amine, et al.
Pubblicazione: (2024)

Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling
di: Wang, Andrew, et al.
Pubblicazione: (2026)

Lil: Less is Less When Applying Post-Training Sparse-Attention Algorithms in Long-Decode Stage
di: Hu, Junhao, et al.
Pubblicazione: (2026)

Circuit Component Reuse Across Tasks in Transformer Language Models
di: Merullo, Jack, et al.
Pubblicazione: (2023)

Language Models Implement Simple Word2Vec-style Vector Arithmetic
di: Merullo, Jack, et al.
Pubblicazione: (2023)

Lightweight Safety Guardrails via Synthetic Data and RL-guided Adversarial Training
di: Ilin, Aleksei, et al.
Pubblicazione: (2025)

LoopTool: Closing the Data-Training Loop for Robust LLM Tool Calls
di: Zhang, Kangning, et al.
Pubblicazione: (2025)

Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky
di: Hathidara, Ashutosh, et al.
Pubblicazione: (2025)

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation
di: Iyer, Vivek, et al.
Pubblicazione: (2025)

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
di: You, Haoran, et al.
Pubblicazione: (2024)

CodecLM: Aligning Language Models with Tailored Synthetic Data
di: Wang, Zifeng, et al.
Pubblicazione: (2024)

LoRA Learns Less and Forgets Less
di: Biderman, Dan, et al.
Pubblicazione: (2024)

Instilling Inductive Biases with Subnetworks
di: Zhang, Enyan, et al.
Pubblicazione: (2023)

Regurgitative Training: The Value of Real Data in Training Large Language Models
di: Zhang, Jinghui, et al.
Pubblicazione: (2024)

Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models
di: Nahin, Shahriar Kabir, et al.
Pubblicazione: (2025)

Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
di: Ba, Yang, et al.
Pubblicazione: (2024)

Training and Evaluating Language Models with Template-based Data Generation
di: Zhang, Yifan
Pubblicazione: (2024)

VRPO: Rethinking Value Modeling for Robust RL Training under Noisy Supervision
di: Zhu, Dingwei, et al.
Pubblicazione: (2025)

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
di: Gan, Zeyu, et al.
Pubblicazione: (2024)

Less is More: Improving LLM Alignment via Preference Data Selection
di: Deng, Xun, et al.
Pubblicazione: (2025)

Transferring Linear Features Across Language Models With Model Stitching
di: Chen, Alan, et al.
Pubblicazione: (2025)

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
di: Li, Kenneth, et al.
Pubblicazione: (2022)

Does Biomedical Training Lead to Better Medical Performance?
di: Dada, Amin, et al.
Pubblicazione: (2024)

Does Refusal Training in LLMs Generalize to the Past Tense?
di: Andriushchenko, Maksym, et al.
Pubblicazione: (2024)

The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling
di: Zhang, Ruochen, et al.
Pubblicazione: (2024)

Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
di: Ulm, Jannek, et al.
Pubblicazione: (2025)

Synthetic Data RL: Task Definition Is All You Need
di: Guo, Yiduo, et al.
Pubblicazione: (2025)

One Swallow Does Not Make a Summer: Understanding Semantic Structures in Embedding Spaces
di: Sun, Yandong, et al.
Pubblicazione: (2025)

Less is More: Local Intrinsic Dimensions of Contextual Language Models
di: Ruppik, Benjamin Matthias, et al.
Pubblicazione: (2025)

Synthetic Data for any Differentiable Target
di: Thrush, Tristan, et al.
Pubblicazione: (2026)

Learn More, Forget Less: A Gradient-Aware Data Selection Approach for LLM
di: Liu, Yibai, et al.
Pubblicazione: (2025)

BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation
di: Zhu, Alan, et al.
Pubblicazione: (2025)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
di: Zhou, Ying, et al.
Pubblicazione: (2024)

Enhancing Clinical Documentation with Synthetic Data: Leveraging Generative Models for Improved Accuracy
di: Biswas, Anjanava, et al.
Pubblicazione: (2024)