Image de couverture de livre

Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhao, Hao, Andriushchenko, Maksym, Croce, Francesco, Flammarion, Nicolas
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Is In-Context Learning Sufficient for Instruction Following in LLMs?
par: Zhao, Hao, et autres
Publié: (2024)

Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
par: Andriushchenko, Maksym, et autres
Publié: (2024)

Does Refusal Training in LLMs Generalize to the Past Tense?
par: Andriushchenko, Maksym, et autres
Publié: (2024)

HalluHard: A Hard Multi-Turn Hallucination Benchmark
par: Fan, Dongyang, et autres
Publié: (2026)

Competition Report: Finding Universal Jailbreak Backdoors in Aligned LLMs
par: Rando, Javier, et autres
Publié: (2024)

SPA: A Simple but Tough-to-Beat Baseline for Knowledge Injection
par: Tang, Kexian, et autres
Publié: (2026)

OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents
par: Kuntz, Thomas, et autres
Publié: (2025)

On the Out-of-Distribution Generalization of Reasoning in Multimodal LLMs for Simple Visual Planning Tasks
par: Neuhaus, Yannic, et autres
Publié: (2026)

Data Repetition Beats Data Scaling in Long-CoT Supervised Fine-Tuning
par: Kopiczko, Dawid J., et autres
Publié: (2026)

Why Do We Need Weight Decay in Modern Deep Learning?
par: D'Angelo, Francesco, et autres
Publié: (2023)

Improved Baselines with Visual Instruction Tuning
par: Liu, Haotian, et autres
Publié: (2023)

Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning
par: Du, Yanrui, et autres
Publié: (2024)

A NotSo Simple Way to Beat Simple Bench
par: Sane, Soham, et autres
Publié: (2024)

FuseLIP: Multimodal Embeddings via Early Fusion of Discrete Tokens
par: Schlarmann, Christian, et autres
Publié: (2025)

On the Adversarial Robustness of Discrete Image Tokenizers
par: Bhagwatkar, Rishika, et autres
Publié: (2026)

Does Instruction Tuning Make LLMs More Consistent?
par: Fierro, Constanza, et autres
Publié: (2024)

SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning
par: Wen, Zhihao, et autres
Publié: (2024)

A Simple Yet Strong Baseline for Long-Term Conversational Memory of LLM Agents
par: Zhou, Sizhe, et autres
Publié: (2025)

(How) Learning Rates Regulate Catastrophic Overtraining
par: Rofin, Mark, et autres
Publié: (2026)

Capability-Based Scaling Trends for LLM-Based Red-Teaming
par: Panfilov, Alexander, et autres
Publié: (2025)

The Atomic Instruction Gap: Instruction-Tuned LLMs Struggle with Simple, Self-Contained Directives
par: Lim, Henry, et autres
Publié: (2025)

Less is More: High-value Data Selection for Visual Instruction Tuning
par: Liu, Zikang, et autres
Publié: (2024)

Instruction Matters: A Simple yet Effective Task Selection for Optimized Instruction Tuning of Specific Tasks
par: Lee, Changho, et autres
Publié: (2024)

Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management
par: Ma, Weitao, et autres
Publié: (2026)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
par: Kiruluta, Andrew
Publié: (2026)

Fine-Tuning on Noisy Instructions: Effects on Generalization and Performance
par: Alajrami, Ahmed, et autres
Publié: (2025)

Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents
par: Talokar, Nivya, et autres
Publié: (2026)

GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
par: Ruan, Zhiwen, et autres
Publié: (2026)

Selective Induction Heads: How Transformers Select Causal Structures In Context
par: D'Angelo, Francesco, et autres
Publié: (2025)

Blind Baselines Beat Membership Inference Attacks for Foundation Models
par: Das, Debeshee, et autres
Publié: (2024)

Balancing Continuous Pre-Training and Instruction Fine-Tuning: Optimizing Instruction-Following in LLMs
par: Jindal, Ishan, et autres
Publié: (2024)

Stealth Fine-Tuning: Efficiently Breaking Alignment in RVLMs Using Self-Generated CoT
par: Yu, Le, et autres
Publié: (2025)

DELIFT: Data Efficient Language model Instruction Fine Tuning
par: Agarwal, Ishika, et autres
Publié: (2024)

ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentation
par: Stahl, Maja, et autres
Publié: (2025)

Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
par: Song, Chiyu, et autres
Publié: (2023)

Low-Resource Fine-Tuning for Multi-Task Structured Information Extraction with a Billion-Parameter Instruction-Tuned Model
par: Chih, Yu Cheng, et autres
Publié: (2025)

Improving Alignment and Robustness with Circuit Breakers
par: Zou, Andy, et autres
Publié: (2024)

Long Context Alignment with Short Instructions and Synthesized Positions
par: Wu, Wenhao, et autres
Publié: (2024)

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
par: Song, Feifan, et autres
Publié: (2024)

DEFT: Distribution-guided Efficient Fine-Tuning for Human Alignment
par: Zhu, Liang, et autres
Publié: (2026)