Imagen de Portada

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Jindal, Ishan, Badrinath, Chandana, Bharti, Pranjal, Vinay, Lakkidi, Sharma, Sachin Dev
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

The Path of Least Resistance: Guiding LLM Reasoning Trajectories with Prefix Consensus
por: Jindal, Ishan, et al.
Publicado: (2026)

Instruction Following without Instruction Tuning
por: Hewitt, John, et al.
Publicado: (2024)

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale
por: Patel, Ajay, et al.
Publicado: (2026)

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models
por: Ramakrishnan, Badrinath, et al.
Publicado: (2025)

Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning
por: Wu, Tianyi, et al.
Publicado: (2025)

The Instruction Gap: LLMs get lost in Following Instruction
por: Tripathi, Vishesh, et al.
Publicado: (2025)

From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
por: Wu, Xuansheng, et al.
Publicado: (2023)

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
por: Xie, Juncheng, et al.
Publicado: (2024)

Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
por: Yang, Jiuding, et al.
Publicado: (2024)

Training with Pseudo-Code for Instruction Following
por: Kumar, Prince, et al.
Publicado: (2025)

UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages
por: Chitale, Pranjal A., et al.
Publicado: (2025)

Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?
por: Zhang, Qinyan, et al.
Publicado: (2025)

A Comparative Analysis of Instruction Fine-Tuning LLMs for Financial Text Classification
por: Fatemi, Sorouralsadat, et al.
Publicado: (2024)

Empowering Persian LLMs for Instruction Following: A Novel Dataset and Training Approach
por: Mokhtarabadi, Hojjat, et al.
Publicado: (2024)

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance
por: Alajrami, Ahmed, et al.
Publicado: (2025)

The Atomic Instruction Gap: Instruction-Tuned LLMs Struggle with Simple, Self-Contained Directives
por: Lim, Henry, et al.
Publicado: (2025)

Mortgage Language Model: Domain-Adaptive Pretraining with Residual Instruction, Alignment Tuning, and Task-Specific Routing
por: Jain, Manish, et al.
Publicado: (2025)

Reverse Preference Optimization for Complex Instruction Following
por: Huang, Xiang, et al.
Publicado: (2025)

Prompt, Translate, Fine-Tune, Re-Initialize, or Instruction-Tune? Adapting LLMs for In-Context Learning in Low-Resource Languages
por: Toukmaji, Christopher, et al.
Publicado: (2025)

The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
por: Wallace, Eric, et al.
Publicado: (2024)

Does Instruction Tuning Make LLMs More Consistent?
por: Fierro, Constanza, et al.
Publicado: (2024)

Instruction-Tuning LLMs for Event Extraction with Annotation Guidelines
por: Srivastava, Saurabh, et al.
Publicado: (2025)

Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning
por: Du, Yanrui, et al.
Publicado: (2024)

GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
por: Ruan, Zhiwen, et al.
Publicado: (2026)

Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
por: He, Yun, et al.
Publicado: (2024)

When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMs
por: Li, Xiaomin, et al.
Publicado: (2025)

DELIFT: Data Efficient Language model Instruction Fine Tuning
por: Agarwal, Ishika, et al.
Publicado: (2024)

ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentation
por: Stahl, Maja, et al.
Publicado: (2025)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
por: Song, Chiyu, et al.
Publicado: (2023)

Thinking LLMs: General Instruction Following with Thought Generation
por: Wu, Tianhao, et al.
Publicado: (2024)

Phased Instruction Fine-Tuning for Large Language Models
por: Pang, Wei, et al.
Publicado: (2024)

SwitchCIT: Switching for Continual Instruction Tuning
por: Wu, Xinbo, et al.
Publicado: (2024)

Is In-Context Learning Sufficient for Instruction Following in LLMs?
por: Zhao, Hao, et al.
Publicado: (2024)

Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks
por: Pantazopoulos, Georgios, et al.
Publicado: (2024)

Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
por: Yousefiramandi, Amirhossein, et al.
Publicado: (2025)

IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization
por: Zhang, Xinghua, et al.
Publicado: (2024)

Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
por: Le, Chenqian, et al.
Publicado: (2025)

Tool Calling for Arabic LLMs: Data Strategies and Instruction Tuning
por: Ersoy, Asim, et al.
Publicado: (2025)

Instruction Pre-Training: Language Models are Supervised Multitask Learners
por: Cheng, Daixuan, et al.
Publicado: (2024)

Instruction Tuning With Loss Over Instructions
por: Shi, Zhengyan, et al.
Publicado: (2024)