Cover Image

On the Effect of Instruction Tuning Loss on Generalization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chatterjee, Anwoy, Renduchintala, H S V N S Kowndinya, Bhatia, Sumit, Chakraborty, Tanmoy
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

POSIX: A Prompt Sensitivity Index For Large Language Models
by: Chatterjee, Anwoy, et al.
Published: (2024)

SMART: Submodular Data Mixture Strategy for Instruction Tuning
by: Renduchintala, H S V N S Kowndinya, et al.
Published: (2024)

Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck?
by: Renduchintala, H S V N S Kowndinya, et al.
Published: (2026)

HIDE and Seek: Detecting Hallucinations in Language Models via Decoupled Representations
by: Chatterjee, Anwoy, et al.
Published: (2025)

Multilingual Language Models Encode Script Over Linguistic Structure
by: Verma, Aastha A K, et al.
Published: (2026)

Instruction Fine-Tuning: Does Prompt Loss Matter?
by: Huerta-Enochian, Mathew, et al.
Published: (2024)

Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems
by: Kumar, Shivani, et al.
Published: (2023)

Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying
by: Renduchintala, Adithya, et al.
Published: (2023)

Consistency Is the Key: Detecting Hallucinations in LLM Generated Text By Checking Inconsistencies About Key Facts
by: Gupta, Raavi, et al.
Published: (2025)

Generative Representational Instruction Tuning
by: Muennighoff, Niklas, et al.
Published: (2024)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)

Multilingual LLMs Inherently Reward In-Language Time-Sensitive Semantic Alignment for Low-Resource Languages
by: Bajpai, Ashutosh, et al.
Published: (2024)

LongForm: Effective Instruction Tuning with Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2023)

Instruction Tuning with Human Curriculum
by: Lee, Bruce W., et al.
Published: (2023)

Instruction Mining: Instruction Data Selection for Tuning Large Language Models
by: Cao, Yihan, et al.
Published: (2023)

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning
by: He, Bingxiang, et al.
Published: (2024)

The Best Instruction-Tuning Data are Those That Fit
by: Zhang, Dylan, et al.
Published: (2025)

Parameter Efficient Instruction Tuning: An Empirical Study
by: He, Pengfei
Published: (2024)

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
by: Li, Ming, et al.
Published: (2024)

Multilingual Instruction Tuning With Just a Pinch of Multilinguality
by: Shaham, Uri, et al.
Published: (2024)

LESS: Selecting Influential Data for Targeted Instruction Tuning
by: Xia, Mengzhou, et al.
Published: (2024)

Instruction Tuning for Large Language Models: A Survey
by: Zhang, Shengyu, et al.
Published: (2023)

MathScale: Scaling Instruction Tuning for Mathematical Reasoning
by: Tang, Zhengyang, et al.
Published: (2024)

From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning
by: Wu, Xuansheng, et al.
Published: (2023)

MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
by: Köksal, Abdullatif, et al.
Published: (2024)

G-Loss: Graph-Guided Fine-Tuning of Language Models
by: Sharma, Aditya, et al.
Published: (2026)

Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters
by: Zou, Junyi
Published: (2026)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Federated Data-Efficient Instruction Tuning for Large Language Models
by: Qin, Zhen, et al.
Published: (2024)

SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe
by: Xiao, Yuxin, et al.
Published: (2024)

Fine-Tuning a Time Series Foundation Model with Wasserstein Loss
by: Chernov, Andrei
Published: (2024)

Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
by: Zhang, Yi-Kai, et al.
Published: (2025)

Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
by: Dima, George-Andrei, et al.
Published: (2025)

Uncertainty-Aware Gradient Signal-to-Noise Data Selection for Instruction Tuning
by: Yuan, Zhihang, et al.
Published: (2026)

BIPEFT: Budget-Guided Iterative Search for Parameter Efficient Fine-Tuning of Large Pretrained Language Models
by: Chang, Aofei, et al.
Published: (2024)

Neural Networks for Learnable and Scalable Influence Estimation of Instruction Fine-Tuning Data
by: Agarwal, Ishika, et al.
Published: (2025)

SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflection
by: Liu, Liangxin, et al.
Published: (2024)

M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
by: Wang, Taowen, et al.
Published: (2024)

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
by: Hui, Tingfeng, et al.
Published: (2024)