Cover Image

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zheng, Yingming, Li, Hanqi, Yu, Kai, Chen, Lu
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)

How does Misinformation Affect Large Language Model Behaviors and Preferences?
by: Peng, Miao, et al.
Published: (2025)

Evolving Subnetwork Training for Large Language Models
by: Li, Hanqi, et al.
Published: (2024)

Fine-tuning can Help Detect Pretraining Data from Large Language Models
by: Zhang, Hengxiang, et al.
Published: (2024)

You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Models
by: He, Wenchong, et al.
Published: (2025)

Semi-supervised Fine-tuning for Large Language Models
by: Luo, Junyu, et al.
Published: (2024)

RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Response
by: Luo, Junyu, et al.
Published: (2024)

Multi-Party Supervised Fine-tuning of Language Models for Multi-Party Dialogue Generation
by: Wang, Xiaoyu, et al.
Published: (2024)

InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models
by: Yan, Yuchen, et al.
Published: (2025)

When Quantization Affects Confidence of Large Language Models?
by: Proskurina, Irina, et al.
Published: (2024)

How Does Response Length Affect Long-Form Factuality
by: Zhao, James Xu, et al.
Published: (2025)

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models
by: Zhao, Liang, et al.
Published: (2024)

On Active Privacy Auditing in Supervised Fine-tuning for White-Box Language Models
by: Sun, Qian, et al.
Published: (2024)

Resonance RoPE: Improving Context Length Generalization of Large Language Models
by: Wang, Suyuchen, et al.
Published: (2024)

Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models
by: Chen, Pin-Yu, et al.
Published: (2025)

KnowTuning: Knowledge-aware Fine-tuning for Large Language Models
by: Lyu, Yougang, et al.
Published: (2024)

Packing Analysis: Packing Is More Appropriate for Large Models or Datasets in Supervised Fine-tuning
by: Wang, Shuhe, et al.
Published: (2024)

Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection
by: Xiong, Feng, et al.
Published: (2024)

Evaluating Social Bias in RAG Systems: When External Context Helps and Reasoning Hurts
by: Parihar, Shweta, et al.
Published: (2026)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

Dhati+: Fine-tuned Large Language Models for Arabic Subjectivity Evaluation
by: Bellaouar, Slimane, et al.
Published: (2025)

Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models
by: Zhou, Andy
Published: (2025)

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
by: Bai, Jiamu, et al.
Published: (2024)

LongSafety: Evaluating Long-Context Safety of Large Language Models
by: Lu, Yida, et al.
Published: (2025)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
by: Wang, Yibo, et al.
Published: (2025)

Fine-tuning a Large Language Model for Automating Computational Fluid Dynamics Simulations
by: Dong, Zhehao, et al.
Published: (2025)

Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
by: Kim, Kyeonghyun, et al.
Published: (2025)

Efficient Length-Generalizable Attention via Causal Retrieval for Long-Context Language Modeling
by: Hu, Xiang, et al.
Published: (2024)

Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments
by: Lu, Qingyu, et al.
Published: (2025)

Advancing Single and Multi-task Text Classification through Large Language Model Fine-tuning
by: Zhao, Hang, et al.
Published: (2024)

Federated Co-tuning Framework for Large and Small Language Models
by: Fan, Tao, et al.
Published: (2024)

Prompting or Fine-tuning? Exploring Large Language Models for Causal Graph Validation
by: Susanti, Yuni, et al.
Published: (2024)

LooGLE: Can Long-Context Language Models Understand Long Contexts?
by: Li, Jiaqi, et al.
Published: (2023)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

When Does Language Transfer Help? Sequential Fine-Tuning for Cross-Lingual Euphemism Detection
by: Sammartino, Julia, et al.
Published: (2025)

Fine-tuned Large Language Models (LLMs): Improved Prompt Injection Attacks Detection
by: Rahman, Md Abdur, et al.
Published: (2024)

SPAFIT: Stratified Progressive Adaptation Fine-tuning for Pre-trained Large Language Models
by: Arora, Samir, et al.
Published: (2024)

PeFoMed: Parameter Efficient Fine-tuning of Multimodal Large Language Models for Medical Imaging
by: He, Jinlong, et al.
Published: (2024)

CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels
by: Wei, Lingxiao, et al.
Published: (2024)