Cover Image

Towards Effective and Efficient Continual Pre-training of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Jie, Chen, Zhipeng, Wang, Jiapeng, Zhou, Kun, Zhu, Yutao, Jiang, Jinhao, Min, Yingqian, Zhao, Wayne Xin, Dou, Zhicheng, Mao, Jiaxin, Lin, Yankai, Song, Ruihua, Xu, Jun, Chen, Xu, Yan, Rui, Wei, Zhewei, Hu, Di, Huang, Wenbing, Wen, Ji-Rong
Format:	Preprint
Published:	2024
Subjects:	Computation and Language 68T50 I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ReasoningLM: Enabling Structural Subgraph Reasoning in Pre-trained Language Models for Question Answering over Knowledge Graph
by: Jiang, Jinhao, et al.
Published: (2023)

Query-oriented Data Augmentation for Session Search
by: Chen, Haonan, et al.
Published: (2024)

R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
by: Song, Huatong, et al.
Published: (2025)

Ground states of planar Schrödinger-Poisson systems with an unbounded potential
by: Du, Miao, et al.
Published: (2024)

On a class of planar Schrödinger-Poisson system with a bounded potential well
by: Du, Miao, et al.
Published: (2023)

MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics
by: Wang, Jiapeng, et al.
Published: (2025)

Session-level Normalization and Click-through Data Enhancement for Session-based Evaluation
by: Chen, Haonan, et al.
Published: (2024)

ICPC-Eval: Probing the Frontiers of LLM Reasoning with Competitive Programming Contests
by: Xu, Shiyi, et al.
Published: (2025)

Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models
by: Sun, Chendong, et al.
Published: (2025)

$FastDoc$: Domain-Specific Fast Continual Pre-training Technique using Document-Level Metadata and Taxonomy
by: Nandy, Abhilash, et al.
Published: (2023)

Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models
by: Sun, Haoxiang, et al.
Published: (2025)

Sentiment Analysis of Spanish Political Party Tweets Using Pre-trained Language Models
by: Song, Chuqiao, et al.
Published: (2024)

Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
by: Fernández-González, Daniel, et al.
Published: (2026)

Shape enumerators of self-dual NRT codes over finite fields
by: Chen, Yin, et al.
Published: (2024)

An invariant-theoretic approach to three weight enumerators of self-dual quantum codes
by: Chen, Yin, et al.
Published: (2024)

Prompt Engineering and the Effectiveness of Large Language Models in Enhancing Human Productivity
by: Anam, Rizal Khoirul
Published: (2025)

User Behavior Simulation with Large Language Model based Agents
by: Wang, Lei, et al.
Published: (2023)

YuLan: An Open-source Large Language Model
by: Zhu, Yutao, et al.
Published: (2024)

DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task
by: Liu, Wenhan, et al.
Published: (2024)

Towards Effective Code-Integrated Reasoning
by: Bai, Fei, et al.
Published: (2025)

WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training
by: Tian, Changxin, et al.
Published: (2025)

The Classification of Graphs on $8$ vertices with Coinciding Zero Forcing number and Maximum Nullity
by: Barrett, Wayne, et al.
Published: (2025)

Seamless and Efficient Interactions within a Mixed-Dimensional Information Space
by: Chen, Chen
Published: (2025)

Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
by: Mai, Jialong, et al.
Published: (2025)

On Steinerberger Curvature and Graph Distance Matrices
by: Chen, Wei-Chia, et al.
Published: (2023)

Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework
by: Chen, Jie, et al.
Published: (2025)

One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models
by: Zhu, Yutao, et al.
Published: (2024)

Decomposing the Entropy-Performance Exchange: The Missing Keys to Unlocking Effective Reinforcement Learning
by: Deng, Jia, et al.
Published: (2025)

Effective lower bounds for spectra of random covers and random unitary bundles
by: Hide, Will
Published: (2023)

More on the full Brouwer Laplacian spectrum conjecture
by: Chen, Xiaodan, et al.
Published: (2025)

Counterexamples to a conjecture on graph inertia
by: Chen, Hongzhang, et al.
Published: (2026)

YuLan-Mini: An Open Data-efficient Language Model
by: Hu, Yiwen, et al.
Published: (2024)

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings
by: Chen, Haonan, et al.
Published: (2025)

Multi-Agent-as-Judge: Aligning LLM-Agent-Based Automated Evaluation with Multi-Dimensional Human Evaluation
by: Chen, Jiaju, et al.
Published: (2025)

Decouple Searching from Training: Scaling Data Mixing via Model Merging for Large Language Model Pre-training
by: Li, Shengrui, et al.
Published: (2026)

Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems
by: Min, Yingqian, et al.
Published: (2024)

On Modules Whose Pure Submodules Are Essential in Direct Summands
by: Gupta, Kaushal, et al.
Published: (2025)

LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaboration
by: Otal, Hakan T., et al.
Published: (2024)

Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Models
by: Land, Sander, et al.
Published: (2024)

A characterization of graphs $G$ with nullity $n(G)-d(G)-1$
by: Xu, Songnian
Published: (2024)