Cover Image

QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretraining

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Fengze, Zhou, Weidong, Liu, Binbin, Yu, Zhimiao, Zhang, Yifan, Lin, Haobin, Yu, Yifeng, Zhang, Bingni, Zhou, Xiaohuan, Wang, Taifeng, Cao, Yong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
by: Liu, Fengze, et al.
Published: (2026)

Target-Oriented Pretraining Data Selection via Neuron-Activated Graph
by: Wang, Zijun, et al.
Published: (2026)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
by: Guo, Ping, et al.
Published: (2025)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
by: Chen, Zhixun, et al.
Published: (2025)

TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training
by: Wang, Yifan, et al.
Published: (2025)

MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languages
by: Han, Wenhan, et al.
Published: (2025)

MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning
by: Li, Xuchen, et al.
Published: (2026)

Mosaic TMEM173 gene mutation R284S leading to STING‐associated vasculopathy with onset in infancy (SAVI)
by: Zhou Yang, et al.
Published: (2024)

MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
by: Yuan, Shen, et al.
Published: (2025)

You are what you watch: How movies and TV affect everything By WalterHickey, New York: Workman Publishing Company. 2023. pp. 240. $22.39 (hbk)
by: Zhimiao Yang
Published: (2024)

Quasi-Large Hole Polarons in BiVO4-Implications for Photocatalysis and Solar Energy Conversion
by: Hao, Zhimeng, et al.
Published: (2025)

Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark
by: Yu, Zhiqi, et al.
Published: (2026)

Improving Romanian LLM Pretraining Data using Diversity and Quality Filtering
by: Negoita, Vlad, et al.
Published: (2025)

Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation
by: Zhou, Yuxuan, et al.
Published: (2024)

Craw4LLM: Efficient Web Crawling for LLM Pretraining
by: Yu, Shi, et al.
Published: (2025)

Harnessing Diversity for Important Data Selection in Pretraining Large Language Models
by: Zhang, Chi, et al.
Published: (2024)

CodingTeachLLM: Empowering LLM's Coding Ability via AST Prior Knowledge
by: Chen, Zhangquan, et al.
Published: (2024)

Stretch Tunable Active Frequency Selective Fabric for Dynamically Switchable Bandstop and Transparency
by: Hao Feng, et al.
Published: (2026)

Contrastive Heliophysical Image Pretraining for Solar Dynamics Observatory Records
by: Shen, Shiyu, et al.
Published: (2025)

Adanonymizer: Interactively Navigating and Balancing the Duality of Privacy and Output Performance in Human-LLM Interaction
by: Zhang, Shuning, et al.
Published: (2024)

Uneventful: Event Semantics for “Qua”
by: Annina Loets
Published: (2024)

MASS: Mathematical Data Selection via Skill Graphs for Pretraining Large Language Models
by: Li, Jiazheng, et al.
Published: (2025)

QuaLLM: An LLM-based Framework to Extract Quantitative Insights from Online Forums
by: Rao, Varun Nagaraj, et al.
Published: (2024)

H-Zero: Cross-Humanoid Locomotion Pretraining Enables Few-shot Novel Embodiment Transfer
by: Lin, Yunfeng, et al.
Published: (2025)

Edge-Based Multimodal Sensor Data Fusion with Vision Language Models (VLMs) for Real-time Autonomous Vehicle Accident Avoidance
by: Yang, Fengze, et al.
Published: (2025)

Dual‐Anion Entropy Engineering of Se–P High‐Entropy Interfaces in HEAs for Optimized H * Binding and Accelerated Hydrogen Evolution
by: Asif Mahmood, et al.
Published: (2026)

Diffusion-Guided Pretraining for Brain Graph Foundation Models
by: Wei, Xinxu, et al.
Published: (2026)

Skip-SCAR: Hardware-Friendly High-Quality Embodied Visual Navigation
by: Liu, Yaotian, et al.
Published: (2024)

ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery
by: Liu, Xin, et al.
Published: (2023)

QuaLITi: Quantum Machine Learning Hardware Selection for Inferencing with Top-Tier Performance
by: Phalak, Koustubh, et al.
Published: (2024)

Agent Assurance in Regulated Financial Services
by: Zhong, Fengze
Published: (2026)

Agent Assurance in Regulated Financial Services
by: Zhong, Fengze
Published: (2026)

Antibiotic exposure alters the honeybee gut microbiota and may interfere with the honeybee behavioral caste transition
by: Zijing Zhang, et al.
Published: (2024)

PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning
by: Wang, Qibin, et al.
Published: (2024)

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length
by: Ma, Xuezhe, et al.
Published: (2024)

Synthesis and Characterization of α,ω‐Dihydroxy Telechelic and Star‐Shaped Poly(γ‐Butyrolactone)s by Organocatalytic Living/Controlled Ring‐Opening Polymerization: Depolymerization and Physical Properties
by: Yihang Liu, et al.
Published: (2025)

Research on an Autonomous UAV Search and Rescue System Based on the Improved
by: Chen, Haobin, et al.
Published: (2024)

WRAP++: Web discoveRy Amplified Pretraining
by: Zhou, Jiang, et al.
Published: (2026)

SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluation
by: Wang, Hui, et al.
Published: (2025)

Arrows of Math Reasoning Data Synthesis for Large Language Models: Diversity, Complexity and Correctness
by: Chen, Sirui, et al.
Published: (2025)