Image de couverture de livre

Learning Dynamics in Continual Pre-Training for Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wang, Xingjin, Tissue, Howe, Wang, Lu, Li, Linjing, Zeng, Daniel Dajun
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaling Law with Learning Rate Annealing
par: Tissue, Howe, et autres
Publié: (2024)

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
par: Zhang, Mozhi, et autres
Publié: (2025)

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
par: Zhuang, Yuchen, et autres
Publié: (2025)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
par: Abbes, Istabrak, et autres
Publié: (2025)

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
par: Wang, Yifei, et autres
Publié: (2024)

DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
par: Fu, Xue-Yong, et autres
Publié: (2025)

Continual Learning of Large Language Models: A Comprehensive Survey
par: Shi, Haizhou, et autres
Publié: (2024)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
par: Wang, Yiping, et autres
Publié: (2025)

Token-Efficient Leverage Learning in Large Language Models
par: Zeng, Yuanhao, et autres
Publié: (2024)

Reinforcement Learning on Pre-Training Data
par: Li, Siheng, et autres
Publié: (2025)

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
par: Zhuo, Zhijian, et autres
Publié: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
par: Ibrahim, Adam, et autres
Publié: (2024)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
par: Schröder, Christopher, et autres
Publié: (2024)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
par: Park, Jungwoo, et autres
Publié: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
par: Zhang, Jun, et autres
Publié: (2025)

Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness
par: Hillebrand, Lars, et autres
Publié: (2024)

Large Language Models Explore by Latent Distilling
par: Zeng, Yuanhao, et autres
Publié: (2026)

Knowledge Boundary Discovery for Large Language Models
par: Wang, Ziquan, et autres
Publié: (2026)

Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
par: Feng, Yujie, et autres
Publié: (2025)

Training Optimal Large Diffusion Language Models
par: Ni, Jinjie, et autres
Publié: (2025)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
par: Shi, Haizhou, et autres
Publié: (2024)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
par: Xi, Zhiheng, et autres
Publié: (2024)

Dynamic Universal Approximation Theory: The Basic Theory for Transformer-based Large Language Models
par: Wang, Wei, et autres
Publié: (2024)

JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
par: Dragomir, Alexandra, et autres
Publié: (2026)

Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models
par: Araujo, Vladimir, et autres
Publié: (2024)

Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLP
par: Zhang, Lihong, et autres
Publié: (2025)

LocMoE: A Low-Overhead MoE for Large Language Model Training
par: Li, Jing, et autres
Publié: (2024)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
par: Jia, Sheng, et autres
Publié: (2025)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
par: Zhao, Shiwan, et autres
Publié: (2026)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
par: Song, Weixi, et autres
Publié: (2023)

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
par: Yi, Rongjie, et autres
Publié: (2024)

Enhancing Hepatopathy Clinical Trial Efficiency: A Secure, Large Language Model-Powered Pre-Screening Pipeline
par: Gui, Xiongbin, et autres
Publié: (2025)

Qalb: Largest State-of-the-Art Urdu Large Language Model for 230M Speakers with Systematic Continued Pre-training
par: Hassan, Muhammad Taimoor, et autres
Publié: (2026)

Sliding Window Attention Training for Efficient Large Language Models
par: Fu, Zichuan, et autres
Publié: (2025)

Interactions Across Blocks in Post-Training Quantization of Large Language Models
par: Shabanovi, Khasmamad, et autres
Publié: (2024)

TableTime: Reformulating Time Series Classification as Training-Free Table Understanding with Large Language Models
par: Wang, Jiahao, et autres
Publié: (2024)

Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning
par: Lin, Xuan, et autres
Publié: (2025)

PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
par: Khan, Rana Muhammad Shahroz, et autres
Publié: (2024)

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
par: Feng, Yujie, et autres
Publié: (2026)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
par: Chen, Mengzhao, et autres
Publié: (2024)