Copertina

TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wang, Yifan, Liu, Binbin, Liu, Fengze, Guo, Yuanfan, Deng, Jiyao, Wu, Xuecheng, Zhou, Weidong, Zhou, Xiaohuan, Wang, Taifeng
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
di: Liu, Fengze, et al.
Pubblicazione: (2026)

QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretraining
di: Liu, Fengze, et al.
Pubblicazione: (2025)

Target-Oriented Pretraining Data Selection via Neuron-Activated Graph
di: Wang, Zijun, et al.
Pubblicazione: (2026)

Exploring Polyglot Harmony: On Multilingual Data Allocation for Large Language Models Pretraining
di: Guo, Ping, et al.
Pubblicazione: (2025)

On semi-simplicity of KMY algebras
di: Alraddadi, Nouf, et al.
Pubblicazione: (2024)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
di: Chen, Zhixun, et al.
Pubblicazione: (2025)

MathMixup: Boosting LLM Mathematical Reasoning with Difficulty-Controllable Data Synthesis and Curriculum Learning
di: Li, Xuchen, et al.
Pubblicazione: (2026)

Ultrasound-CLIP: Semantic-Aware Contrastive Pre-training for Ultrasound Image-Text Understanding
di: Jin, Jiayun, et al.
Pubblicazione: (2026)

MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
di: Yuan, Shen, et al.
Pubblicazione: (2025)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
di: Zhou, Fan, et al.
Pubblicazione: (2024)

RegMix: Data Mixture as Regression for Language Model Pre-training
di: Liu, Qian, et al.
Pubblicazione: (2024)

RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
di: Bi, Baolong, et al.
Pubblicazione: (2025)

Quasi-Large Hole Polarons in BiVO4-Implications for Photocatalysis and Solar Energy Conversion
di: Hao, Zhimeng, et al.
Pubblicazione: (2025)

HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training
di: Wu, Xuecheng, et al.
Pubblicazione: (2025)

Beyond Sunk Costs: Boosting LLM Pre-training Efficiency via Orthogonal Growth of Mixture-of-Experts
di: Wang, Ruizhe, et al.
Pubblicazione: (2025)

Data Darwinism Part I: Unlocking the Value of Scientific Data for Pre-training
di: Qin, Yiwei, et al.
Pubblicazione: (2026)

Controlled Walnut Shelling Through Structure‐Guided Pre‐Notches: Optimizing Kernel Integrity and Process Efficiency
di: Quanxin Cui, et al.
Pubblicazione: (2026)

Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences
di: Yu, Liu, et al.
Pubblicazione: (2025)

Multi-modal Multi-task Pre-training for Improved Point Cloud Understanding
di: Liu, Liwen, et al.
Pubblicazione: (2025)

MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
di: Wang, Jiapeng, et al.
Pubblicazione: (2025)

Event Camera Data Dense Pre-training
di: Yang, Yan, et al.
Pubblicazione: (2023)

Optimization Model of Steel‐Prestressed Concrete Hybrid Wind Turbine Tower: Using a Combined Differential Whale Optimization Algorithm
di: Wei Xu, et al.
Pubblicazione: (2025)

Continuous Multi-Task Pre-training for Malicious URL Detection and Webpage Classification
di: Li, Yujie, et al.
Pubblicazione: (2024)

Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning
di: Wang, Huiyi, et al.
Pubblicazione: (2024)

GraphGPT: Generative Pre-trained Graph Eulerian Transformer
di: Zhao, Qifang, et al.
Pubblicazione: (2023)

Pre-trained Visual Dynamics Representations for Efficient Policy Learning
di: Luo, Hao, et al.
Pubblicazione: (2024)

YEZE at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization via Heterogeneous Ensembling
di: Guo, Fengze, et al.
Pubblicazione: (2026)

MuBench: Assessment of Multilingual Capabilities of Large Language Models Across 61 Languages
di: Han, Wenhan, et al.
Pubblicazione: (2025)

Temporal-Consistent Video Restoration with Pre-trained Diffusion Models
di: Wang, Hengkang, et al.
Pubblicazione: (2025)

Thermal Management Design and Key Technology Validation for PandaX Underground Experiment
di: Zhan, Tao, et al.
Pubblicazione: (2024)

Pre-training Everywhere: Parameter-Efficient Fine-Tuning for Medical Image Analysis via Target Parameter Pre-training
di: Lei, Xingliang, et al.
Pubblicazione: (2024)

Influence-driven Curriculum Learning for Pre-training on Limited Data
di: Schoenegger, Loris, et al.
Pubblicazione: (2025)

SA-WavLM: Speaker-Aware Self-Supervised Pre-training for Mixture Speech
di: Lin, Jingru, et al.
Pubblicazione: (2024)

Enhancing Adversarial Transferability in Visual-Language Pre-training Models via Local Shuffle and Sample-based Attack
di: Liu, Xin, et al.
Pubblicazione: (2025)

CUT: Pruning Pre-Trained Multi-Task Models into Compact Models for Edge Devices
di: Zhou, Jingxuan, et al.
Pubblicazione: (2025)

Fight Poison with Poison: Enhancing Robustness in Few-shot Machine-Generated Text Detection with Adversarial Training
di: Duan, Wenjing, et al.
Pubblicazione: (2026)

Dual‐Anion Entropy Engineering of Se–P High‐Entropy Interfaces in HEAs for Optimized H * Binding and Accelerated Hydrogen Evolution
di: Asif Mahmood, et al.
Pubblicazione: (2026)

Read the Docs Before Rewriting: Equip Rewriter with Domain Knowledge via Continual Pre-training
di: Wang, Qi, et al.
Pubblicazione: (2025)

Advancing Multi-grained Alignment for Contrastive Language-Audio Pre-training
di: Li, Yiming, et al.
Pubblicazione: (2024)

The Influence of Task and Group Disparities over Users' Attitudes Toward Using Large Language Models for Psychotherapy
di: He, Qihang, et al.
Pubblicazione: (2024)