Cover Image

A Survey on LLM Mid-Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Tu, Chengying, Zhang, Xuemiao, Weng, Rongxiang, Li, Rumei, Zhang, Chen, Bai, Yang, Yan, Hongfei, Wang, Jingang, Cai, Xunliang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Large-Scale Diverse Synthesis for Mid-Training
by: Zhang, Xuemiao, et al.
Published: (2025)

LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points
by: Zhang, Xuemiao, et al.
Published: (2025)

Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
by: Zhang, Xuemiao, et al.
Published: (2025)

FIRE: Flexible Integration of Data Quality Ratings for Effective Pre-Training
by: Xu, Liangyu, et al.
Published: (2025)

Unlocking Implicit Experience: Synthesizing Tool-Use Trajectories from Text
by: Xu, Zhihao, et al.
Published: (2026)

FRAME: Boosting LLMs with A Four-Quadrant Multi-Stage Pretraining Strategy
by: Zhang, Xuemiao, et al.
Published: (2025)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
by: Zhang, Xuemiao, et al.
Published: (2025)

Libra: Assessing and Improving Reward Model by Learning to Think
by: Zhou, Meng, et al.
Published: (2025)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
by: Bai, Yang, et al.
Published: (2026)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
by: Diao, Muxi, et al.
Published: (2024)

Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism
by: Liu, Jiahao, et al.
Published: (2024)

Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
by: Tang, Hongyin, et al.
Published: (2024)

LANG: Reinforcement Learning for Multilingual Reasoning with Language-Adaptive Hint Guidance
by: Fan, Yuchun, et al.
Published: (2026)

Mid-Training of Large Language Models: A Survey
by: Mo, Kaixiang, et al.
Published: (2025)

Enhancing LLMs via High-Knowledge Data Selection
by: Duan, Feiyu, et al.
Published: (2025)

Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training
by: Wang, Zhijun, et al.
Published: (2025)

FIRP: Faster LLM inference via future intermediate representation prediction
by: Wu, Pengfei, et al.
Published: (2024)

ReMamba: Equip Mamba with Effective Long-Sequence Modeling
by: Yuan, Danlong, et al.
Published: (2024)

Look Before You Leap: Enhancing Attention and Vigilance Regarding Harmful Content with GuidelineLLM
by: Zhang, Shaoqing, et al.
Published: (2024)

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
by: Shao, Ruichen, et al.
Published: (2025)

Dynamic Fisher-weighted Model Merging via Bayesian Optimization
by: Lee, Sanwoo, et al.
Published: (2025)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
by: Xi, Zhiheng, et al.
Published: (2024)

SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
by: Xi, Xiangyu, et al.
Published: (2025)

Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment
by: Zhang, Jiazheng, et al.
Published: (2025)

A Preliminary Study on the Promises and Challenges of Native Top-$k$ Sparse Attention
by: Xiu, Di, et al.
Published: (2025)

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
by: Liu, Kaiyuan, et al.
Published: (2026)

Graph-Structured Speculative Decoding
by: Gong, Zhuocheng, et al.
Published: (2024)

SpanNorm: Reconciling Training Stability and Performance in Deep Transformers
by: Wang, Chao, et al.
Published: (2026)

GUI-CIDER: Mid-training GUI Agents via Causal Internalization and Density-aware Exemplar Reselection
by: Wu, Zheng, et al.
Published: (2026)

Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
by: Wu, Pengfei, et al.
Published: (2024)

A Survey on Data Selection for LLM Instruction Tuning
by: Zhang, Bolin, et al.
Published: (2024)

Efficient Context Scaling with LongCat ZigZag Attention
by: Zhang, Chen, et al.
Published: (2025)

Selecting Demonstrations for Many-Shot In-Context Learning via Gradient Matching
by: Zhang, Jianfei, et al.
Published: (2025)

NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables
by: Wang, Lanrui, et al.
Published: (2025)

FABSVer: Faster Training and Better Self-Verification for LLM Mathematical Reasoning
by: Pan, Haihui, et al.
Published: (2026)

Reinforcement Mid-Training
by: Tian, Yijun, et al.
Published: (2025)

ReMiT: RL-Guided Mid-Training for Iterative LLM Evolution
by: Huang, Junjie, et al.
Published: (2026)

Prejudge-Before-Think: Enhancing Large Language Models at Test-Time by Process Prejudge Reasoning
by: Wang, Jianing, et al.
Published: (2025)

Causal Autoregressive Diffusion Language Model
by: Ruan, Junhao, et al.
Published: (2026)