Cover Image

Mortgage Language Model: Domain-Adaptive Pretraining with Residual Instruction, Alignment Tuning, and Task-Specific Routing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Jain, Manish, Ponnambalam, Satheesh Kumar, Faroz, Salman, Lns, Chandrakanth, Sharma, Vinay
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Domain-Adaptive Continued Pre-Training of Small Language Models
by: Faroz, Salman
Published: (2025)

DisEmbed: Transforming Disease Understanding through Embeddings
by: Faroz, Salman
Published: (2024)

Enhancing reasoning accuracy in large language models during inference time
by: Sharma, Vinay, et al.
Published: (2026)

Sensitivity Analysis On Loss Landscape
by: Faroz, Salman
Published: (2024)

BrailleLLM: Braille Instruction Tuning with Large Language Models for Braille Domain Tasks
by: Huang, Tianyuan, et al.
Published: (2025)

DS$^2$-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning
by: Xu, Ruiyao, et al.
Published: (2026)

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs
by: Jindal, Ishan, et al.
Published: (2024)

Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models
by: Parmar, Jupinder, et al.
Published: (2024)

Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly
by: Gao, Changjiang, et al.
Published: (2024)

Instruction Matters: A Simple yet Effective Task Selection for Optimized Instruction Tuning of Specific Tasks
by: Lee, Changho, et al.
Published: (2024)

Progressive Residual Warmup for Language Model Pretraining
by: Chen, Tianhao, et al.
Published: (2026)

AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning
by: Cui, Jiaxi, et al.
Published: (2024)

CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
by: Tang, Jun-Tao, et al.
Published: (2026)

AlignTune: Modular Toolkit for Post-Training Alignment of Large Language Models
by: Lyngkhoi, R E Zera Marveen, et al.
Published: (2026)

Fine-Tuned Language Models for Domain-Specific Summarization and Tagging
by: Wang, Jun, et al.
Published: (2025)

Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling
by: Grangier, David, et al.
Published: (2024)

SOCIALITE-LLAMA: An Instruction-Tuned Model for Social Scientific Tasks
by: Dey, Gourab, et al.
Published: (2024)

APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference
by: Zhao, Bowen, et al.
Published: (2024)

Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models
by: Setty, Vinay
Published: (2024)

Exploring the Benefits of Domain-Pretraining of Generative Large Language Models for Chemistry
by: Acharya, Anurag, et al.
Published: (2024)

RAFT: Adapting Language Model to Domain Specific RAG
by: Zhang, Tianjun, et al.
Published: (2024)

How Important is Domain Specificity in Language Models and Instruction Finetuning for Biomedical Relation Extraction?
by: Brokman, Aviv, et al.
Published: (2024)

Set the Clock: Temporal Alignment of Pretrained Language Models
by: Zhao, Bowen, et al.
Published: (2024)

Domain-Specific Pretraining of Language Models: A Comparative Study in the Medical Field
by: Kerner, Tobias
Published: (2024)

Table Comprehension in Building Codes using Vision Language Models and Domain-Specific Fine-Tuning
by: Aqib, Mohammad, et al.
Published: (2025)

Fine-Tuning Language Models for Ethical Ambiguity: A Comparative Study of Alignment with Human Responses
by: Senthilkumar, Pranav, et al.
Published: (2024)

Healing Powers of BERT: How Task-Specific Fine-Tuning Recovers Corrupted Language Models
by: Han, Shijie, et al.
Published: (2024)

Chain-of-Instructions: Compositional Instruction Tuning on Large Language Models
by: Hayati, Shirley Anugrah, et al.
Published: (2024)

InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment
by: Wang, Jianing, et al.
Published: (2024)

Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning
by: Susnjak, Teo, et al.
Published: (2024)

LAET: A Layer-wise Adaptive Ensemble Tuning Framework for Pretrained Language Models
by: Ahad, Jawad Ibn, et al.
Published: (2025)

From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
by: Wang, Fei, et al.
Published: (2024)

ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
by: Wu, Yang, et al.
Published: (2024)

LayerRoute: Input-Conditioned Adaptive Layer Skipping via LoRA Fine-Tuning for Agentic Language Models
by: Sikdar, Prateek Kumar
Published: (2026)

TuneVLSeg: Prompt Tuning Benchmark for Vision-Language Segmentation Models
by: Adhikari, Rabin, et al.
Published: (2024)

Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
by: Han, Xudong, et al.
Published: (2025)

A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment
by: Corbeil, Jean-Philippe, et al.
Published: (2025)

IKnow: Instruction-Knowledge-Aware Continual Pretraining for Effective Domain Adaptation
by: Zhang, Tianyi, et al.
Published: (2025)

The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities
by: Bigoulaeva, Irina, et al.
Published: (2025)

InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
by: Wang, Boxin, et al.
Published: (2023)