Cover Image

Can Continual Pre-training Bridge the Performance Gap between General-purpose and Specialized Language Models in the Medical Domain?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Doll, Niclas, Buschhoff, Jasper Schulze, Satheesh, Shalaka, Abdelwahab, Hammam, Allende-Cid, Héctor, Klug, Katrin
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

GG-BBQ: German Gender Bias Benchmark for Question Answering
by: Satheesh, Shalaka, et al.
Published: (2025)

Tokenizer Choice For LLM Training: Negligible or Crucial?
by: Ali, Mehdi, et al.
Published: (2023)

Evaluation von Verwaltungsmodernisierung
by: Buschhoff, Christian
Published: (2020)

Efficient Continual Pre-training by Mitigating the Stability Gap
by: Guo, Yiduo, et al.
Published: (2024)

Detecting Linguistic Indicators for Stereotype Assessment with Large Language Models
by: Görge, Rebekka, et al.
Published: (2025)

Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences
by: Yu, Liu, et al.
Published: (2025)

Can Diffusion Models Bridge the Domain Gap in Cardiac MR Imaging?
by: Wong, Xin Ci, et al.
Published: (2025)

Finetuning Pre-trained Model with Limited Data for LiDAR-based 3D Object Detection by Bridging Domain Gaps
by: Jang, Jiyun, et al.
Published: (2024)

Not‐so‐Freeway: A Relational Approach to Checkpoints and Conflict in Northeast India
by: Shalaka Thakur
Published: (2026)

Push button parliament–why India needs a non-partisan, recorded vote system
by: Patil Shalaka
Published: (2011)

Metaplastic breast carcinoma with extensive osseous differentiation
by: Shalaka Khade
Published: (2021)

Microcontroller Based Self-Regulating Devices in Enclosed Environments
by: Shalaka Subrahmanyam
Published: (2014)

Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation
by: Görge, Rebekka, et al.
Published: (2025)

Bridging Domain Gaps between Pretrained Multimodal Models and Recommendations
by: Zhang, Wenyu, et al.
Published: (2025)

The Gaps between Pre-train and Downstream Settings in Bias Evaluation and Debiasing
by: Kaneko, Masahiro, et al.
Published: (2024)

Inductive Graph Alignment Prompt: Bridging the Gap between Graph Pre-training and Inductive Fine-tuning From Spectral Perspective
by: Yan, Yuchen, et al.
Published: (2024)

Exploring Pre-trained General-purpose Audio Representations for Heart Murmur Detection
by: Niizumi, Daisuke, et al.
Published: (2024)

Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data?
by: Liu, Che, et al.
Published: (2024)

Bridging the Gap: Integrating Pre-trained Speech Enhancement and Recognition Models for Robust Speech Recognition
by: Wang, Kuan-Chen, et al.
Published: (2024)

Exploring the Limits of Model Compression in LLMs: A Knowledge Distillation Study on QA Tasks
by: Datta, Joyeeta, et al.
Published: (2025)

Named Entity Recognition for Konkani Speech
by: Shalaka Naik Dessai
Published: (2025)

Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens
by: Salahuddin, Salahuddin, et al.
Published: (2025)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
by: Yang, Kailai, et al.
Published: (2025)

SONAR: Self-Distilled Continual Pre-training for Domain Adaptive Audio Representation
by: Zhang, Yizhou, et al.
Published: (2025)

Efficient Continual Pre-training for Building Domain Specific Large Language Models
by: Xie, Yong, et al.
Published: (2023)

Bridging the Gap: Unravelling Local Government Data Sharing Barriers in Estonia and Beyond
by: Soosaar, Katrin Rajamäe, et al.
Published: (2024)

Domain Pre-training Impact on Representations
by: Gonzalez-Gutierrez, Cesar, et al.
Published: (2025)

Bridging the Gap between Continuous and Informative Discrete Representations by Random Product Quantization
by: Li, Xueqing, et al.
Published: (2025)

Velocitune: A Velocity-based Dynamic Domain Reweighting Method for Continual Pre-training
by: Luo, Zheheng, et al.
Published: (2024)

MaskedCLIP: Bridging the Masked and CLIP Space for Semi-Supervised Medical Vision-Language Pre-training
by: Zhu, Lei, et al.
Published: (2025)

Exploiting Pre-trained Encoder-Decoder Transformers for Sequence-to-Sequence Constituent Parsing
by: Fernández-González, Daniel, et al.
Published: (2026)

The Algebraic and Geometric Classification of Compatible Pre-Lie Algebras
by: Abdelwahab, Hani, et al.
Published: (2024)

Bridging the Inter-Domain Gap through Low-Level Features for Cross-Modal Medical Image Segmentation
by: Lyu, Pengfei, et al.
Published: (2025)

Bridging the Gap between the Student and the Library.
by: Egan, Philip J.
Published: (1992)

Gap distribution of $\sqrt{n} \,\mathrm{mod}\, 1$ and the circle method
by: Radziwiłł, Maksym, et al.
Published: (2024)

MergeOcc: Bridge the Domain Gap between Different LiDARs for Robust Occupancy Prediction
by: Xu, Zikun, et al.
Published: (2024)

Domain-Adaptive Pre-training of Self-Supervised Foundation Models for Medical Image Classification in Gastrointestinal Endoscopy
by: Roth, Marcel, et al.
Published: (2024)

Read the Docs Before Rewriting: Equip Rewriter with Domain Knowledge via Continual Pre-training
by: Wang, Qi, et al.
Published: (2025)

Construction of Domain-specified Japanese Large Language Model for Finance through Continual Pre-training
by: Hirano, Masanori, et al.
Published: (2024)

Mecellem Models: Turkish Models Trained from Scratch and Continually Pre-trained for the Legal Domain
by: Uğur, Özgür, et al.
Published: (2026)