Cover Image

How Useful is Continued Pre-Training for Generative Unsupervised Domain Adaptation?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Uppaal, Rheeya, Li, Yixuan, Hu, Junjie
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains
by: Kakkar, Ishita, et al.
Published: (2026)

Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity
by: Uppaal, Rheeya, et al.
Published: (2024)

Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking
by: Uppaal, Rheeya, et al.
Published: (2025)

Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts
by: Boudin, Florian, et al.
Published: (2024)

Continual Pre-Training is (not) What You Need in Domain Adaption
by: Chen, Pin-Er, et al.
Published: (2025)

Improving Bilingual Capabilities of Language Models to Support Diverse Linguistic Practices in Education
by: Syamkumar, Anand, et al.
Published: (2024)

Domain-Adaptive Continued Pre-Training of Small Language Models
by: Faroz, Salman
Published: (2025)

Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic
by: Zhao, Xingyu, et al.
Published: (2026)

How Do Large Language Models Learn Concepts During Continual Pre-Training?
by: Yao, Barry Menglong, et al.
Published: (2026)

Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain
by: Uğur, Özgür, et al.
Published: (2026)

DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation
by: Chandradevan, Ramraj, et al.
Published: (2024)

PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
by: Goffinet, Etienne, et al.
Published: (2025)

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
by: Fujii, Kazuki, et al.
Published: (2024)

Domain Adaptation of Llama3-70B-Instruct through Continual Pre-Training and Model Merging: A Comprehensive Evaluation
by: Siriwardhana, Shamane, et al.
Published: (2024)

DAdEE: Unsupervised Domain Adaptation in Early Exit PLMs
by: Bajpai, Divya Jyoti, et al.
Published: (2024)

D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models
by: Que, Haoran, et al.
Published: (2024)

Learning Beyond the Surface: How Far Can Continual Pre-Training with LoRA Enhance LLMs' Domain-Specific Insight Learning?
by: Pezeshkpour, Pouya, et al.
Published: (2025)

Unmasking the Imposters: How Censorship and Domain Adaptation Affect the Detection of Machine-Generated Tweets
by: Tuck, Bryan E., et al.
Published: (2024)

How Far Can Unsupervised RLVR Scale LLM Training?
by: He, Bingxiang, et al.
Published: (2026)

Breaking Language Barriers: Cross-Lingual Continual Pre-Training at Scale
by: Zheng, Wenzhen, et al.
Published: (2024)

AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation
by: Meng, Rui, et al.
Published: (2022)

Language Model as an Annotator: Unsupervised Context-aware Quality Phrase Generation
by: Zhang, Zhihao, et al.
Published: (2023)

DACIP-RC: Domain Adaptive Continual Instruction Pre-Training via Reading Comprehension on Business Conversations
by: Khasanova, Elena, et al.
Published: (2025)

Freeze Deep, Train Shallow: Interpretable Layer Allocation for Continued Pre-Training
by: Wu, Yu-Hang, et al.
Published: (2026)

CTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teacher
by: Liu, Tianlun, et al.
Published: (2025)

How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes
by: Bhasin, Harmon, et al.
Published: (2024)

Organize the Web: Constructing Domains Enhances Pre-Training Data Curation
by: Wettig, Alexander, et al.
Published: (2025)

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
by: Luo, Zheheng, et al.
Published: (2024)

Domain-Adaptive Pre-Training for Arabic Aspect-Based Sentiment Analysis: A Comparative Study of Domain Adaptation and Fine-Tuning Strategies
by: Alyami, Salha, et al.
Published: (2025)

Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training
by: Reddy, Arun, et al.
Published: (2023)

DACP: Domain-Adaptive Continual Pre-Training of Large Language Models for Phone Conversation Summarization
by: Fu, Xue-Yong, et al.
Published: (2025)

How Well Do LLMs Identify Cultural Unity in Diversity?
by: Li, Jialin, et al.
Published: (2024)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
by: Xie, Yong, et al.
Published: (2023)

How to Select Pre-Trained Code Models for Reuse? A Learning Perspective
by: Bi, Zhangqian, et al.
Published: (2025)

Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysis
by: Boughorbel, Sabri, et al.
Published: (2024)

Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?
by: Doll, Niclas, et al.
Published: (2026)

MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASR
by: Damianos, Dimitrios, et al.
Published: (2025)

Reinforcement Pre-Training
by: Dong, Qingxiu, et al.
Published: (2025)

Reinforced Curriculum Pre-Alignment for Domain-Adaptive VLMs
by: Yan, Yuming, et al.
Published: (2026)

How Far Are We from Genuinely Useful Deep Research Agents?
by: Zhang, Dingling, et al.
Published: (2025)