Image de couverture de livre

Characterizing Model Behavior Under Synthetic Data Training: An Empirical Study Across Scales and Mixing Ratios

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Du, Y., Wu, G., Tang, G., Wang, W., Fan, Q.
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
par: Michaelov, James A., et autres
Publié: (2025)

An Empirical Study of Validating Synthetic Data for Formula Generation
par: Singh, Usneek, et autres
Publié: (2024)

Can Small Language Models Use What They Retrieve? An Empirical Study of Retrieval Utilization Across Model Scale
par: Pandey, Sanchit
Publié: (2026)

Cognitive Decision Routing in Large Language Models: When to Think Fast, When to Think Slow
par: Du, Y., et autres
Publié: (2025)

On the Diversity of Synthetic Data and its Impact on Training Large Language Models
par: Chen, Hao, et autres
Publié: (2024)

Scaling Speech-Text Pre-training with Synthetic Interleaved Data
par: Zeng, Aohan, et autres
Publié: (2024)

Mitigating Gambling-Like Risk-Taking Behaviors in Large Language Models: A Behavioral Economics Approach to AI Safety
par: Du, Y.
Publié: (2025)

Reliability Under Randomness: An Empirical Analysis of Sparse and Dense Language Models Across Decoding Temperatures
par: Grover, Kabir
Publié: (2026)

Synthetic Data Generation for Training Diversified Commonsense Reasoning Models
par: Zhang, Tianhui, et autres
Publié: (2026)

Prescriptive Scaling Laws for Data Constrained Training
par: Lovelace, Justin, et autres
Publié: (2026)

Scaling Laws of Synthetic Data for Language Models
par: Qin, Zeyu, et autres
Publié: (2025)

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs
par: Kang, Feiyang, et autres
Publié: (2024)

What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA
par: He, Xinjie, et autres
Publié: (2026)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
par: Guo, Xu, et autres
Publié: (2026)

Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study
par: Cui, Menglong, et autres
Publié: (2025)

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
par: Patel, Ajay, et autres
Publié: (2026)

A Study on Training and Developing Large Language Models for Behavior Tree Generation
par: Li, Fu, et autres
Publié: (2024)

OpenCharacter: Training Customizable Role-Playing LLMs with Large-Scale Synthetic Personas
par: Wang, Xiaoyang, et autres
Publié: (2025)

Assessing Judging Bias in Large Reasoning Models: An Empirical Study
par: Wang, Qian, et autres
Publié: (2025)

Empirical Characterization of Rationale Stability Under Controlled Perturbations for Explainable Pattern Recognition
par: Sakib, Abu Noman Md, et autres
Publié: (2026)

LLM Circuit Analyses Are Consistent Across Training and Scale
par: Tigges, Curt, et autres
Publié: (2024)

LLM-to-Speech: A Synthetic Data Pipeline for Training Dialectal Text-to-Speech Models
par: Khamis, Ahmed Khaled, et autres
Publié: (2026)

Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures
par: Wang, Junxuan, et autres
Publié: (2024)

Train a Unified Multimodal Data Quality Classifier with Synthetic Data
par: Wang, Weizhi, et autres
Publié: (2025)

Synthetic Sandbox for Training Machine Learning Engineering Agents
par: Zhou, Yuhang, et autres
Publié: (2026)

Does RLHF Scale? Exploring the Impacts From Data, Model, and Method
par: Hou, Zhenyu, et autres
Publié: (2024)

SK-VQA: Synthetic Knowledge Generation at Scale for Training Context-Augmented Multimodal LLMs
par: Su, Xin, et autres
Publié: (2024)

From Data to Behavior: Predicting Unintended Model Behaviors Before Training
par: Wang, Mengru, et autres
Publié: (2026)

Gradient-Based LoRA Rank Allocation Under GRPO: An Empirical Study
par: Sawant, Yash Ganpat
Publié: (2026)

Characterizing the Behavior of Training Mamba-based State Space Models on GPUs
par: Baruah, Trinayan, et autres
Publié: (2025)

Scaling Arabic Medical Chatbots Using Synthetic Data: Enhancing Generative AI with Synthetic Patient Records
par: Allam, Abdulrahman, et autres
Publié: (2025)

Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources
par: Li, Zihao, et autres
Publié: (2025)

Does Training on Synthetic Data Make Models Less Robust?
par: Zhang, Lingze, et autres
Publié: (2025)

Semi-Synthetic Parallel Data for Translation Quality Estimation: A Case Study of Dataset Building for an Under-Resourced Language Pair
par: Siani, Assaf, et autres
Publié: (2026)

Quantifying LLM Biases Across Instruction Boundary in Mixed Question Forms
par: Ling, Zipeng, et autres
Publié: (2025)

Evaluating Language Models as Synthetic Data Generators
par: Kim, Seungone, et autres
Publié: (2024)

Scaling Low-Resource MT via Synthetic Data Generation with LLMs
par: de Gibert, Ona, et autres
Publié: (2025)

Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation
par: Kartik, Kartik, et autres
Publié: (2024)

Training and Evaluating with Human Label Variation: An Empirical Study
par: Kurniawan, Kemal, et autres
Publié: (2025)

DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
par: Zhou, Ying, et autres
Publié: (2024)