Copertina

On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Charlie, Neubig, Graham, Yue, Xiang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Go-Browse: Training Web Agents with Structured Exploration
di: Gandhi, Apurva, et al.
Pubblicazione: (2025)

Learning to Reason as Action Abstractions with Scalable Mid-Training RL
di: Zhang, Shenao, et al.
Pubblicazione: (2025)

CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models
di: Sheikh, Zaid, et al.
Pubblicazione: (2024)

Training Versatile Coding Agents in Synthetic Environments
di: Zhu, Yiqi, et al.
Pubblicazione: (2025)

BehaviorBox: Automated Discovery of Fine-Grained Performance Differences Between Language Models
di: Tjuatja, Lindia, et al.
Pubblicazione: (2025)

ReMiT: RL-Guided Mid-Training for Iterative LLM Evolution
di: Huang, Junjie, et al.
Pubblicazione: (2026)

On the Optimal Reasoning Length for RL-Trained Language Models
di: Nohara, Daisuke, et al.
Pubblicazione: (2026)

Demystifying Long Chain-of-Thought Reasoning in LLMs
di: Yeo, Edward, et al.
Pubblicazione: (2025)

Mid-Training of Large Language Models: A Survey
di: Mo, Kaixiang, et al.
Pubblicazione: (2025)

Training Task Experts through Retrieval Based Distillation
di: Ge, Jiaxin, et al.
Pubblicazione: (2024)

VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge
di: Song, Yueqi, et al.
Pubblicazione: (2025)

Synthetic Pre-Pre-Training Improves Language Model Robustness to Noisy Pre-Training Data
di: Guo, Xu, et al.
Pubblicazione: (2026)

What Is Missing in Multilingual Visual Reasoning and How to Fix It
di: Song, Yueqi, et al.
Pubblicazione: (2024)

An Incomplete Loop: Instruction Inference, Instruction Following, and In-context Learning in Language Models
di: Liu, Emmy, et al.
Pubblicazione: (2024)

SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
di: Chen, Hardy, et al.
Pubblicazione: (2025)

Reinforcement Mid-Training
di: Tian, Yijun, et al.
Pubblicazione: (2025)

Training Software Engineering Agents and Verifiers with SWE-Gym
di: Pan, Jiayi, et al.
Pubblicazione: (2024)

A Survey on LLM Mid-Training
di: Tu, Chengying, et al.
Pubblicazione: (2025)

FastCuRL: Curriculum Reinforcement Learning with Stage-wise Context Scaling for Efficient Training R1-like Reasoning Models
di: Song, Mingyang, et al.
Pubblicazione: (2025)

Exploring Forgetting in Large Language Model Pre-Training
di: Liao, Chonghua, et al.
Pubblicazione: (2024)

Checklists Are Better Than Reward Models For Aligning Language Models
di: Viswanathan, Vijay, et al.
Pubblicazione: (2025)

Pause or Fabricate? Training Language Models for Grounded Reasoning
di: Qiu, Yiwen, et al.
Pubblicazione: (2026)

Beyond Two-Stage Training: Cooperative SFT and RL for LLM Reasoning
di: Chen, Liang, et al.
Pubblicazione: (2025)

Evolution of Concepts in Language Model Pre-Training
di: Ge, Xuyang, et al.
Pubblicazione: (2025)

Scaling Evaluation-time Compute with Reasoning Models as Evaluators
di: Kim, Seungone, et al.
Pubblicazione: (2025)

Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale
di: Zheng, Wenzhen, et al.
Pubblicazione: (2024)

Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysis
di: Boughorbel, Sabri, et al.
Pubblicazione: (2024)

LoopRPT: Reinforcement Pre-Training for Looped Language Models
di: Tang, Guo, et al.
Pubblicazione: (2026)

MiniPLM: Knowledge Distillation for Pre-Training Language Models
di: Gu, Yuxian, et al.
Pubblicazione: (2024)

Analysing The Impact of Sequence Composition on Language Model Pre-Training
di: Zhao, Yu, et al.
Pubblicazione: (2024)

Instruction Pre-Training: Language Models are Supervised Multitask Learners
di: Cheng, Daixuan, et al.
Pubblicazione: (2024)

ConcEPT: Concept-Enhanced Pre-Training for Language Models
di: Wang, Xintao, et al.
Pubblicazione: (2024)

Overtrained Language Models Are Harder to Fine-Tune
di: Springer, Jacob Mitchell, et al.
Pubblicazione: (2025)

Training Language Models to Reason Efficiently
di: Arora, Daman, et al.
Pubblicazione: (2025)

Large-Scale Diverse Synthesis for Mid-Training
di: Zhang, Xuemiao, et al.
Pubblicazione: (2025)

Evaluating Language Models as Synthetic Data Generators
di: Kim, Seungone, et al.
Pubblicazione: (2024)

Verifiable by Design: Aligning Language Models to Quote from Pre-Training Data
di: Zhang, Jingyu, et al.
Pubblicazione: (2024)

Breaking the Script Barrier in Multilingual Pre-Trained Language Models with Transliteration-Based Post-Training Alignment
di: Xhelili, Orgest, et al.
Pubblicazione: (2024)

Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
di: Huan, Maggie, et al.
Pubblicazione: (2025)

Improving Automatic Summarization of Radiology Reports through Mid-Training of Large Language Models
di: Lyu, Mengxian, et al.
Pubblicazione: (2026)