Buchumschlag

Learning Dynamics in Continual Pre-Training for Large Language Models

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Wang, Xingjin, Tissue, Howe, Wang, Lu, Li, Linjing, Zeng, Daniel Dajun
Format:	Preprint
Veröffentlicht:	2025
Schlagworte:	Computation and Language Artificial Intelligence Machine Learning
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Ähnliche Einträge

Scaling Law with Learning Rate Annealing
von: Tissue, Howe, et al.
Veröffentlicht: (2024)

Domain2Vec: Vectorizing Datasets to Find the Optimal Data Mixture without Training
von: Zhang, Mozhi, et al.
Veröffentlicht: (2025)

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
von: Zhuang, Yuchen, et al.
Veröffentlicht: (2025)

Revisiting Replay and Gradient Alignment for Continual Pre-Training of Large Language Models
von: Abbes, Istabrak, et al.
Veröffentlicht: (2025)

Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
von: Wang, Yifei, et al.
Veröffentlicht: (2024)

DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
von: Fu, Xue-Yong, et al.
Veröffentlicht: (2025)

Continual Learning of Large Language Models: A Comprehensive Survey
von: Shi, Haizhou, et al.
Veröffentlicht: (2024)

Reinforcement Learning for Reasoning in Large Language Models with One Training Example
von: Wang, Yiping, et al.
Veröffentlicht: (2025)

Token-Efficient Leverage Learning in Large Language Models
von: Zeng, Yuanhao, et al.
Veröffentlicht: (2024)

Reinforcement Learning on Pre-Training Data
von: Li, Siheng, et al.
Veröffentlicht: (2025)

Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models
von: Zhuo, Zhijian, et al.
Veröffentlicht: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
von: Ibrahim, Adam, et al.
Veröffentlicht: (2024)

Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language Models
von: Schröder, Christopher, et al.
Veröffentlicht: (2024)

Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models
von: Park, Jungwoo, et al.
Veröffentlicht: (2025)

Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models
von: Zhang, Jun, et al.
Veröffentlicht: (2025)

Pointer-Guided Pre-Training: Infusing Large Language Models with Paragraph-Level Contextual Awareness
von: Hillebrand, Lars, et al.
Veröffentlicht: (2024)

Large Language Models Explore by Latent Distilling
von: Zeng, Yuanhao, et al.
Veröffentlicht: (2026)

Knowledge Boundary Discovery for Large Language Models
von: Wang, Ziquan, et al.
Veröffentlicht: (2026)

Recurrent Knowledge Identification and Fusion for Language Model Continual Learning
von: Feng, Yujie, et al.
Veröffentlicht: (2025)

Training Optimal Large Diffusion Language Models
von: Ni, Jinjie, et al.
Veröffentlicht: (2025)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
von: Shi, Haizhou, et al.
Veröffentlicht: (2024)

Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning
von: Xi, Zhiheng, et al.
Veröffentlicht: (2024)

Dynamic Universal Approximation Theory: The Basic Theory for Transformer-based Large Language Models
von: Wang, Wei, et al.
Veröffentlicht: (2024)

JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
von: Dragomir, Alexandra, et al.
Veröffentlicht: (2026)

Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models
von: Araujo, Vladimir, et al.
Veröffentlicht: (2024)

Federated Learning with Layer Skipping: Efficient Training of Large Language Models for Healthcare NLP
von: Zhang, Lihong, et al.
Veröffentlicht: (2025)

LocMoE: A Low-Overhead MoE for Large Language Model Training
von: Li, Jing, et al.
Veröffentlicht: (2024)

Training Large Language Models To Reason In Parallel With Global Forking Tokens
von: Jia, Sheng, et al.
Veröffentlicht: (2025)

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
von: Zhao, Shiwan, et al.
Veröffentlicht: (2026)

Sparse is Enough in Fine-tuning Pre-trained Large Language Models
von: Song, Weixi, et al.
Veröffentlicht: (2023)

PhoneLM:an Efficient and Capable Small Language Model Family through Principled Pre-training
von: Yi, Rongjie, et al.
Veröffentlicht: (2024)

Enhancing Hepatopathy Clinical Trial Efficiency: A Secure, Large Language Model-Powered Pre-Screening Pipeline
von: Gui, Xiongbin, et al.
Veröffentlicht: (2025)

Qalb: Largest State-of-the-Art Urdu Large Language Model for 230M Speakers with Systematic Continued Pre-training
von: Hassan, Muhammad Taimoor, et al.
Veröffentlicht: (2026)

Sliding Window Attention Training for Efficient Large Language Models
von: Fu, Zichuan, et al.
Veröffentlicht: (2025)

Interactions Across Blocks in Post-Training Quantization of Large Language Models
von: Shabanovi, Khasmamad, et al.
Veröffentlicht: (2024)

TableTime: Reformulating Time Series Classification as Training-Free Table Understanding with Large Language Models
von: Wang, Jiahao, et al.
Veröffentlicht: (2024)

Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning
von: Lin, Xuan, et al.
Veröffentlicht: (2025)

PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
von: Khan, Rana Muhammad Shahroz, et al.
Veröffentlicht: (2024)

FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
von: Feng, Yujie, et al.
Veröffentlicht: (2026)

EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
von: Chen, Mengzhao, et al.
Veröffentlicht: (2024)