Cover Image

Domain-Specific Pretraining of Language Models: A Comparative Study in the Medical Field

Fuente: arXiv

Saved in:

Bibliographic Details
Main Author:	Kerner, Tobias
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence Computation and Language I.2.6; I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)

Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scale
by: Kaushal, Ayush, et al.
Published: (2024)

Beyond Hallucinations: A Composite Score for Measuring Reliability in Open-Source Large Language Models
by: Salla, Rohit Kumar, et al.
Published: (2025)

Prototype Transformer: Towards Language Model Architectures Interpretable by Design
by: Yordanov, Yordan, et al.
Published: (2026)

Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models
by: Zhang, Gongbo, et al.
Published: (2026)

Cognitive Load Limits in Large Language Models: Benchmarking Multi-Hop Reasoning
by: Adapala, Sai Teja Reddy
Published: (2025)

Painless Activation Steering: An Automated, Lightweight Approach for Post-Training Large Language Models
by: Cui, Sasha, et al.
Published: (2025)

The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models
by: Liu, Ming
Published: (2026)

Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding
by: Feng, Qi, et al.
Published: (2025)

The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies
by: Garcia, Gabriel
Published: (2026)

Language as a Wave Phenomenon: Semantic Phase Locking and Interference in Neural Networks
by: Yıldırım, Alper, et al.
Published: (2025)

Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies
by: Liu, Ming
Published: (2026)

Model Collapse as Cultural Evolution
by: Guo, Dongxin, et al.
Published: (2026)

The Data Efficiency Frontier of Financial Foundation Models: Scaling Laws from Continued Pretraining
by: Ponnock, Jesse
Published: (2025)

The Drill-Down and Fabricate Test (DDFT): A Protocol for Measuring Epistemic Robustness in Language Models
by: Baxi, Rahul
Published: (2025)

Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation
by: Shafique, Muhammad Ali, et al.
Published: (2025)

EasyMath: A 0-shot Math Benchmark for SLMs
by: Karki, Drishya, et al.
Published: (2025)

Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation
by: Lyu, Bohan, et al.
Published: (2024)

Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features
by: Cho, Seonglae, et al.
Published: (2026)

In-Context Fixation: When Demonstrated Labels Override Semantics in Few-Shot Classification
by: Liu, Ming
Published: (2026)

No Free Swap: Protocol-Dependent Layer Redundancy in Transformers
by: Garcia, Gabriel
Published: (2026)

Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries
by: Dragoi, Marius, et al.
Published: (2025)

Revisiting Intermediate-Layer Matching in Knowledge Distillation: Layer-Selection Strategy Doesn't Matter (Much)
by: Yu, Zony, et al.
Published: (2025)

Counterfactual Likelihood Tests for Indirect Influence in Private Reasoning Channels
by: Lorup, Alexander Boesgaard
Published: (2026)

TIAR: Trajectory-Informed Advantage Reweighting for LLM Abstention Learning
by: Pan, Muyu, et al.
Published: (2026)

Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
by: Kumar, Sachin
Published: (2026)

Structured Prompt Optimization Meets Reinforcement Learning for Global and Local Interpretability over Complex Text
by: Zhou, Tianyang, et al.
Published: (2026)

AMEL: Accumulated Message Effects on LLM Judgments
by: Temkit, Sid-Ali
Published: (2026)

Forget Attention: Importance-Aware Attention Is All You Need
by: Shin, Soohyeong, et al.
Published: (2026)

Enhancing Burmese News Classification with Kolmogorov-Arnold Network Head Fine-tuning
by: Aung, Thura, et al.
Published: (2025)

MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledge
by: Lan, Guangchen, et al.
Published: (2025)

Dodo: Dynamic Contextual Compression for Decoder-only LMs
by: Qin, Guanghui, et al.
Published: (2023)

CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
by: Cho, Seonglae, et al.
Published: (2025)

Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning
by: Xu, Shuyao, et al.
Published: (2025)

DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory
by: Zhou, Wenxuan, et al.
Published: (2025)

Contextual Integrity in LLMs via Reasoning and Reinforcement Learning
by: Lan, Guangchen, et al.
Published: (2025)

Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking
by: Zhang, Liangliang, et al.
Published: (2025)

Generalizing Numerical Reasoning in Table Data through Operation Sketches and Self-Supervised Learning
by: Cho, Hanjun, et al.
Published: (2026)

The Instability of Safety: How Random Seeds and Temperature Expose Inconsistent LLM Refusal Behavior
by: Larsen, Erik
Published: (2025)

Alternating Reinforcement Learning with Contextual Rubric Rewards: Beyond the Scalarization Strategy
by: Lan, Guangchen, et al.
Published: (2026)