Cover Image

Data Mixing Can Induce Phase Transitions in Knowledge Acquisition

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gu, Xinran, Lyu, Kaifeng, Li, Jiazheng, Zhang, Jingzhao
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
by: Lyu, Kaifeng, et al.
Published: (2024)

SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection
by: Tang, Kexian, et al.
Published: (2026)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
by: Xu, Jing, et al.
Published: (2024)

AcquisitionSynthesis: Targeted Data Generation using Acquisition Functions
by: Agarwal, Ishika, et al.
Published: (2026)

The Power of Power Law: Asymmetry Enables Compositional Reasoning
by: Wang, Zixuan, et al.
Published: (2026)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
by: Luo, Kairong, et al.
Published: (2025)

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
by: Lyu, Kaifeng, et al.
Published: (2023)

Bridging the Knowledge Void: Inference-time Acquisition of Unfamiliar Programming Languages for Coding Tasks
by: Shen, Chen, et al.
Published: (2026)

When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
by: Wang, Shaowen, et al.
Published: (2025)

Semantic Refinement with LLMs for Graph Representations
by: Thapaliya, Safal, et al.
Published: (2025)

BiMix: A Bivariate Data Mixing Law for Language Model Pretraining
by: Ge, Ce, et al.
Published: (2024)

Long-Short Distance Graph Neural Networks and Improved Curriculum Learning for Emotion Recognition in Conversation
by: Li, Xinran, et al.
Published: (2025)

Can Interpretation Predict Behavior on Unseen Data?
by: Li, Victoria R., et al.
Published: (2025)

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
by: Luo, Kairong, et al.
Published: (2025)

Always Learning, Always Mixing: Efficient and Simple Data Mixing All The Time
by: Hu, Michael Y., et al.
Published: (2026)

The Geometric Anatomy of Capability Acquisition in Transformers
by: Billa, Jayadev
Published: (2026)

Rethinking Data Mixing from the Perspective of Large Language Models
by: Xu, Yuanjian, et al.
Published: (2026)

You Can Generate It Again: Data-to-Text Generation with Verification and Correction Prompting
by: Ren, Xuan, et al.
Published: (2023)

Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
by: Yousuf, Raquib Bin, et al.
Published: (2025)

Lying Is Just a Phase: The Hidden Alignment Transition in Language Model Scaling
by: Amin, Adil
Published: (2026)

Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
by: Ye, Jiasheng, et al.
Published: (2024)

Can Large Language Models Infer Causation from Correlation?
by: Jin, Zhijing, et al.
Published: (2023)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
by: Yang, Kailai, et al.
Published: (2025)

Olmix: A Framework for Data Mixing Throughout LM Development
by: Chen, Mayee F., et al.
Published: (2026)

When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions
by: Xia, Wei, et al.
Published: (2026)

Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning
by: Ye, Jiasheng, et al.
Published: (2023)

How Much Can We Forget about Data Contamination?
by: Bordt, Sebastian, et al.
Published: (2024)

SciAtlas: A Large-Scale Knowledge Graph for Automated Scientific Research
by: Qiao, Shuofei, et al.
Published: (2026)

Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Merging
by: Yang, Jinluan, et al.
Published: (2025)

Does Machine Unlearning Truly Remove Knowledge?
by: Chen, Haokun, et al.
Published: (2025)

Aioli: A Unified Optimization Framework for Language Model Data Mixing
by: Chen, Mayee F., et al.
Published: (2024)

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs
by: Kang, Feiyang, et al.
Published: (2024)

KOCO-BENCH: Can Large Language Models Leverage Domain Knowledge in Software Development?
by: Jiang, Xue, et al.
Published: (2026)

Human Inspired Progressive Alignment and Comparative Learning for Grounded Word Acquisition
by: Bao, Yuwei, et al.
Published: (2023)

MixAtlas: Uncertainty-aware Data Mixture Optimization for Multimodal LLM Midtraining
by: Wen, Bingbing, et al.
Published: (2026)

PCMind-2.1-Kaiyuan-2B Technical Report
by: Luo, Kairong, et al.
Published: (2025)

AdaptThink: Reasoning Models Can Learn When to Think
by: Zhang, Jiajie, et al.
Published: (2025)

MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropies
by: Zhang, Shiyue, et al.
Published: (2023)

Reasoning is Periodicity? Improving Large Language Models Through Effective Periodicity Modeling
by: Dong, Yihong, et al.
Published: (2025)