Image de couverture de livre

Efficient Continual Pre-training for Building Domain Specific Large Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xie, Yong, Aggarwal, Karan, Ahmad, Aitzaz
Format:	Preprint
Publié:	2023
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Controlled Automatic Task-Specific Synthetic Data Generation for Hallucination Detection
par: Xie, Yong, et autres
Publié: (2024)

Learning from Contrastive Prompts: Automated Optimization and Adaptation
par: Li, Mingqi, et autres
Publié: (2024)

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training
par: Hirano, Masanori, et autres
Publié: (2024)

Towards Effective and Efficient Continual Pre-training of Large Language Models
par: Chen, Jie, et autres
Publié: (2024)

Examining Forgetting in Continual Pre-training of Aligned Large Language Models
par: Li, Chen-An, et autres
Publié: (2024)

Vi-Mistral-X: Building a Vietnamese Language Model with Advanced Continual Pre-training
par: Vo, James
Publié: (2024)

Efficiently Building a Domain-Specific Large Language Model from Scratch: A Case Study of a Classical Chinese Large Language Model
par: Li, Shen, et autres
Publié: (2025)

STEP: Staged Parameter-Efficient Pre-training for Large Language Models
par: Yano, Kazuki, et autres
Publié: (2025)

RASL: Retrieval Augmented Schema Linking for Massive Database Text-to-SQL
par: Eben, Jeffrey, et autres
Publié: (2025)

Efficient Continual Pre-training of LLMs for Low-resource Languages
par: Nag, Arijit, et autres
Publié: (2024)

Simple and Scalable Strategies to Continually Pre-train Large Language Models
par: Ibrahim, Adam, et autres
Publié: (2024)

Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain
par: Uğur, Özgür, et autres
Publié: (2026)

Efficient Continual Pre-training by Mitigating the Stability Gap
par: Guo, Yiduo, et autres
Publié: (2024)

DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
par: Fu, Xue-Yong, et autres
Publié: (2025)

Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings
par: Sharma, Kartik, et autres
Publié: (2025)

Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?
par: Doll, Niclas, et autres
Publié: (2026)

Model Merging in Pre-training of Large Language Models
par: Li, Yunshui, et autres
Publié: (2025)

Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study
par: Wu, Di, et autres
Publié: (2022)

On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation
par: Wu, Di, et autres
Publié: (2024)

Scaling Agents via Continual Pre-training
par: Su, Liangcai, et autres
Publié: (2025)

Advances in Pre-trained Language Models for Domain-Specific Text Classification: A Systematic Review
par: Rostam, Zhyar Rzgar K., et autres
Publié: (2025)

Efficient Language Adaptive Pre-training: Extending State-of-the-Art Large Language Models for Polish
par: Ruciński, Szymon
Publié: (2024)

On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neurons
par: Kojima, Takeshi, et autres
Publié: (2024)

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
par: Que, Haoran, et autres
Publié: (2024)

From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan
par: Yang, Lei, et autres
Publié: (2025)

Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Models
par: Ma, Shengjie, et autres
Publié: (2025)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
par: Wang, Shaobo, et autres
Publié: (2026)

Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model
par: Aggarwal, Divyanshu, et autres
Publié: (2024)

Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
par: Wang, Xixi, et autres
Publié: (2025)

G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks
par: Wan, Zhongwei, et autres
Publié: (2022)

Cross-layer Attention Sharing for Pre-trained Large Language Models
par: Mu, Yongyu, et autres
Publié: (2024)

Domain Pre-training Impact on Representations
par: Gonzalez-Gutierrez, Cesar, et autres
Publié: (2025)

Building Domain-Specific Small Language Models via Guided Data Generation
par: Kumar, Aman, et autres
Publié: (2025)

Domain-Adaptive Continued Pre-Training of Small Language Models
par: Faroz, Salman
Publié: (2025)

Table Comprehension in Building Codes using Vision Language Models and Domain-Specific Fine-Tuning
par: Aqib, Mohammad, et autres
Publié: (2025)

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
par: Luo, Zheheng, et autres
Publié: (2024)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
par: Qian, Chen, et autres
Publié: (2024)

MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector
par: Fu, Wenjie, et autres
Publié: (2024)

MAPLE: Multilingual Evaluation of Parameter Efficient Finetuning of Large Language Models
par: Aggarwal, Divyanshu, et autres
Publié: (2024)

Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation
par: Zhu, Hongyin, et autres
Publié: (2021)