Copertina

Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Li, Shengrui, Zhao, Fei, Zhao, Kaiyan, Ye, Jieying, Liu, Haifeng, Shi, Fangcheng, Xie, Zheyong, Hu, Yao, Cao, Shaosheng
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence 68T50 I.2.7
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Towards Effective and Efficient Continual Pre-training of Large Language Models
di: Chen, Jie, et al.
Pubblicazione: (2024)

Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
di: Fernández-González, Daniel, et al.
Pubblicazione: (2026)

Prompt Engineering and the Effectiveness of Large Language Models in Enhancing Human Productivity
di: Anam, Rizal Khoirul
Pubblicazione: (2025)

Multi-Model Synthetic Training for Mission-Critical Small Language Models
di: Platt, Nolan, et al.
Pubblicazione: (2025)

Mixing Times of Glauber Dynamics on Masked Language Models
di: Sana, Suvadip, et al.
Pubblicazione: (2026)

DEM: Distribution Edited Model for Training with Mixed Data Distributions
di: Ram, Dhananjay, et al.
Pubblicazione: (2024)

$FastDoc$: Domain-Specific Fast Continual Pre-training Technique using Document-Level Metadata and Taxonomy
di: Nandy, Abhilash, et al.
Pubblicazione: (2023)

NurValues: Real-World Nursing Values Evaluation for Large Language Models in Clinical Context
di: Yao, Ben, et al.
Pubblicazione: (2025)

Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation
di: Liu, Aiwei, et al.
Pubblicazione: (2024)

Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial
di: Johnson, Warren, et al.
Pubblicazione: (2026)

Examining Linguistic Shifts in Academic Writing Before and After the Launch of ChatGPT: A Study on Preprint Papers
di: Bao, Tong, et al.
Pubblicazione: (2025)

The Superalignment of Superhuman Intelligence with Large Language Models
di: Huang, Minlie, et al.
Pubblicazione: (2024)

Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning
di: Seo, Yeongbin, et al.
Pubblicazione: (2024)

CLMN: Concept based Language Models via Neural Symbolic Reasoning
di: Yang, Yibo
Pubblicazione: (2025)

An Unforgeable Publicly Verifiable Watermark for Large Language Models
di: Liu, Aiwei, et al.
Pubblicazione: (2023)

Profiling German Text Simplification with Interpretable Model-Fingerprints
di: Klöser, Lars, et al.
Pubblicazione: (2026)

Exploring State Tracking Capabilities of Large Language Models
di: Rezaee, Kiamehr, et al.
Pubblicazione: (2025)

Evaluating Pixel Language Models on Non-Standardized Languages
di: Muñoz-Ortiz, Alberto, et al.
Pubblicazione: (2024)

LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures
di: Wu, Yuhang, et al.
Pubblicazione: (2026)

Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
di: Zhang, Zhehao, et al.
Pubblicazione: (2025)

A Survey of Text Watermarking in the Era of Large Language Models
di: Liu, Aiwei, et al.
Pubblicazione: (2023)

Empowering Tabular Data Preparation with Language Models: Why and How?
di: Chen, Mengshi, et al.
Pubblicazione: (2025)

Distilling Large Language Models for Efficient Clinical Information Extraction
di: Vedula, Karthik S., et al.
Pubblicazione: (2024)

Adaptive Steering and Remasking for Safe Generation in Diffusion Language Models
di: Lee, Yejin, et al.
Pubblicazione: (2026)

Accurate Retraining-free Pruning for Pretrained Encoder-based Language Models
di: Park, Seungcheol, et al.
Pubblicazione: (2023)

On the Robustness of Document-Level Relation Extraction Models to Entity Name Variations
di: Meng, Shiao, et al.
Pubblicazione: (2024)

ADE: Adaptive Dictionary Embeddings -- Scaling Multi-Anchor Representations to Large Language Models
di: Demirci, Orhan, et al.
Pubblicazione: (2026)

Beyond Many-Shot Translation: Scaling In-Context Demonstrations For Low-Resource Machine Translation
di: Salim, Luis Frentzen, et al.
Pubblicazione: (2026)

PairCFR: Enhancing Model Training on Paired Counterfactually Augmented Data through Contrastive Learning
di: Qiu, Xiaoqi, et al.
Pubblicazione: (2024)

Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Models
di: Park, Seungcheol, et al.
Pubblicazione: (2025)

Copyright Detection in Large Language Models: An Ethical Approach to Generative AI Development
di: Szczecina, David, et al.
Pubblicazione: (2025)

Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information
di: Park, Seungcheol, et al.
Pubblicazione: (2025)

Setting Standards in Turkish NLP: TR-MMLU for Large Language Model Evaluation
di: Bayram, M. Ali, et al.
Pubblicazione: (2024)

d-TreeRPO: Towards More Reliable Policy Optimization for Diffusion Language Models
di: Pan, Leyi, et al.
Pubblicazione: (2025)

A Knowledge Enhanced Learning and Semantic Composition Model for Multi-Claim Fact Checking
di: Wang, Shuai, et al.
Pubblicazione: (2021)

Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Survey
di: Fang, Xi, et al.
Pubblicazione: (2024)

WeDLM: Reconciling Diffusion Language Models with Standard Causal Attention for Fast Inference
di: Liu, Aiwei, et al.
Pubblicazione: (2025)

Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models
di: Pan, Leyi, et al.
Pubblicazione: (2025)

The Paradox of Poetic Intent in Back-Translation: Evaluating the Quality of Large Language Models in Chinese Translation
di: Weigang, Li, et al.
Pubblicazione: (2025)

Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing
di: Wang, Zhilin, et al.
Pubblicazione: (2025)