Cover Image

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Kailai, Liu, Xiao, Ji, Lei, Li, Hao, Liang, Xiao, Liu, Zhiwei, Gong, Yeyun, Cheng, Peng, Yang, Mao
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
by: Luo, Zheheng, et al.
Published: (2024)

Overcoming Vocabulary Mismatch: Vocabulary-agnostic Teacher Guided Language Modeling
by: Shin, Haebin, et al.
Published: (2025)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Improving Data and Reward Design for Scientific Reasoning in Large Language Models
by: Chen, Zijie, et al.
Published: (2026)

TL;DR: Too Long, Do Re-weighting for Efficient LLM Reasoning Compression
by: Li, Zhong-Zhi, et al.
Published: (2025)

Task Oriented In-Domain Data Augmentation
by: Liang, Xiao, et al.
Published: (2024)

Rumor Detection by Multi-task Suffix Learning based on Time-series Dual Sentiments
by: Liu, Zhiwei, et al.
Published: (2025)

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation
by: Zhu, Hongyin, et al.
Published: (2021)

PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization
by: Ma, Xinbei, et al.
Published: (2023)

LayerNorm Induces Recency Bias in Transformer Decoders
by: Kim, Junu, et al.
Published: (2025)

Learning from the Best, Differently: A Diversity-Driven Rethinking on Data Selection
by: He, Hongyi, et al.
Published: (2025)

Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning
by: Park, Sungjin, et al.
Published: (2024)

RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information
by: Liu, Zhiwei, et al.
Published: (2024)

Pre-training, Fine-tuning and Re-ranking: A Three-Stage Framework for Legal Question Answering
by: Ni, Shiwen, et al.
Published: (2024)

ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy
by: Yang, Zonghan, et al.
Published: (2024)

Optimizing Large Language Model Training Using FP4 Quantization
by: Wang, Ruizhe, et al.
Published: (2025)

ConspEmoLLM: Conspiracy Theory Detection Using an Emotion-Based Large Language Model
by: Liu, Zhiwei, et al.
Published: (2024)

Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding
by: Wang, Cheng, et al.
Published: (2024)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
by: Li, Haoling, et al.
Published: (2024)

AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages
by: Yu, Hao, et al.
Published: (2026)

SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
by: Xi, Xiangyu, et al.
Published: (2025)

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training
by: Liu, Lei, et al.
Published: (2025)

DeepThink: Aligning Language Models with Domain-Specific User Intents
by: Li, Yang, et al.
Published: (2025)

EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis
by: Liu, Zhiwei, et al.
Published: (2024)

Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
by: Cheng, Runxi, et al.
Published: (2026)

DataMan: Data Manager for Pre-training Large Language Models
by: Peng, Ru, et al.
Published: (2025)

Beyond Length: Quantifying Long-Range Information for Long-Context LLM Pretraining Data
by: Deng, Haoran, et al.
Published: (2025)

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training
by: He, Nan, et al.
Published: (2024)

Scaling Agents via Continual Pre-training
by: Su, Liangcai, et al.
Published: (2025)

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
by: Yang, Lei, et al.
Published: (2025)

Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning
by: Huang, Yiming, et al.
Published: (2024)

Advancing Language Multi-Agent Learning with Credit Re-Assignment for Interactive Environment Generalization
by: He, Zhitao, et al.
Published: (2025)

Revealing the Learning Dynamics of Long-Context Continual Pre-training
by: Liang, Yupu, et al.
Published: (2026)

Beyond Fixed Length: Bucket Pre-training is All You Need
by: Yang, Qing, et al.
Published: (2024)

RegMix: Data Mixture as Regression for Language Model Pre-training
by: Liu, Qian, et al.
Published: (2024)

Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
by: Yang, Hao, et al.
Published: (2025)

Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training
by: Zhang, Haode, et al.
Published: (2023)

Can Pre-trained Language Models Understand Chinese Humor?
by: Chen, Yuyan, et al.
Published: (2024)

MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs
by: Liu, Zhiwei, et al.
Published: (2025)

Emotion Detection for Misinformation: A Review
by: Liu, Zhiwei, et al.
Published: (2023)