Cover Image

Instruction Diversity Drives Generalization To Unseen Tasks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Dylan, Wang, Justin, Charton, Francois
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

$\textbf{Only-IF}$:Revealing the Decisive Effect of Instruction Diversity on Generalization
by: Zhang, Dylan, et al.
Published: (2024)

From Symbolic Tasks to Code Generation: Diversification Yields Better Task Performers
by: Zhang, Dylan, et al.
Published: (2024)

A Tale of Tails: Model Collapse as a Change of Scaling Laws
by: Dohmatob, Elvis, et al.
Published: (2024)

The Best Instruction-Tuning Data are Those That Fit
by: Zhang, Dylan, et al.
Published: (2025)

Iteration Head: A Mechanistic Study of Chain-of-Thought
by: Cabannes, Vivien, et al.
Published: (2024)

Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilities
by: Dai, Qirun, et al.
Published: (2025)

Learning to Generalize Unseen Domains via Multi-Source Meta Learning for Text Classification
by: Hu, Yuxuan, et al.
Published: (2024)

Learning the greatest common divisor: explaining transformer predictions
by: Charton, François
Published: (2023)

RAG-Instruct: Boosting LLMs with Diverse Retrieval-Augmented Instructions
by: Liu, Wanlong, et al.
Published: (2024)

TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Data
by: Zhang, Jipeng, et al.
Published: (2024)

Can Interpretation Predict Behavior on Unseen Data?
by: Li, Victoria R., et al.
Published: (2025)

Generative Representational Instruction Tuning
by: Muennighoff, Niklas, et al.
Published: (2024)

Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters
by: Zou, Junyi
Published: (2026)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
by: Indurthi, Sathish Reddy, et al.
Published: (2024)

ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
by: Wu, Yang, et al.
Published: (2024)

Int2Int: a framework for mathematics with transformers
by: Charton, François
Published: (2025)

Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
by: Pham, Bao, et al.
Published: (2026)

Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning
by: Ye, Jiasheng, et al.
Published: (2023)

MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses
by: Yang, Zonglin, et al.
Published: (2024)

On the Effect of Instruction Tuning Loss on Generalization
by: Chatterjee, Anwoy, et al.
Published: (2025)

ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs
by: Chen, Justin Chih-Yao, et al.
Published: (2023)

Is Escalation Worth It? A Decision-Theoretic Characterization of LLM Cascades
by: Bouchard, Dylan
Published: (2026)

OmniStruct: Universal Text-to-Structure Generation across Diverse Schemas
by: Huang, James Y., et al.
Published: (2025)

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages
by: Wang, Zhilin, et al.
Published: (2025)

Does Pre-trained Language Model Actually Infer Unseen Links in Knowledge Graph Completion?
by: Sakai, Yusuke, et al.
Published: (2023)

Emergent properties with repeated examples
by: Charton, François, et al.
Published: (2024)

Instruction Tuning for Large Language Models: A Survey
by: Zhang, Shengyu, et al.
Published: (2023)

Instruction Mining: Instruction Data Selection for Tuning Large Language Models
by: Cao, Yihan, et al.
Published: (2023)

PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
by: Goffinet, Etienne, et al.
Published: (2025)

MathScale: Scaling Instruction Tuning for Mathematical Reasoning
by: Tang, Zhengyang, et al.
Published: (2024)

Diversity-oriented Data Augmentation with Large Language Models
by: Wang, Zaitian, et al.
Published: (2025)

Diversity Boosts AI-Generated Text Detection
by: Basani, Advik Raj, et al.
Published: (2025)

Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following
by: Yang, Brian, et al.
Published: (2024)

RLAC: Reinforcement Learning with Adversarial Critic for Free-Form Generation Tasks
by: Wu, Mian, et al.
Published: (2025)

A Zero-shot and Few-shot Study of Instruction-Finetuned Large Language Models Applied to Clinical and Biomedical Tasks
by: Labrak, Yanis, et al.
Published: (2023)

Learning Representations for Reasoning: Generalizing Across Diverse Structures
by: Zhu, Zhaocheng
Published: (2024)

SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation
by: Divekar, Abhishek, et al.
Published: (2024)

Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification
by: Bouchard, Dylan, et al.
Published: (2026)

Contrastive Instruction Tuning
by: Yan, Tianyi Lorena, et al.
Published: (2024)

SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
by: Suvarna, Ashima, et al.
Published: (2026)