Cover Image

Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lu, Yuheng, Bai, ZiMeng, Yuan, Caixia, Jiang, Huixing, Wang, Xiaojie
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Controlled Low-Rank Adaptation with Subspace Regularization for Continued Training on Large Language Models
by: Lu, Yuheng, et al.
Published: (2024)

On the Loss of Context-awareness in General Instruction Fine-tuning
by: Wang, Yihan, et al.
Published: (2024)

Fine-tuning Large Language Models with Sequential Instructions
by: Hu, Hanxu, et al.
Published: (2024)

Phased Instruction Fine-Tuning for Large Language Models
by: Pang, Wei, et al.
Published: (2024)

Demystifying Instruction Mixing for Fine-tuning Large Language Models
by: Wang, Renxi, et al.
Published: (2023)

From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models
by: He, Qianyu, et al.
Published: (2024)

Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach
by: Zhou, Meng, et al.
Published: (2024)

EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models
by: Zou, Tao, et al.
Published: (2025)

Fine-Tuning is Subgraph Search: A New Lens on Learning Dynamics
by: Li, Yueyan, et al.
Published: (2025)

You Only Fine-tune Once: Many-Shot In-Context Fine-Tuning for Large Language Models
by: He, Wenchong, et al.
Published: (2025)

InsCL: A Data-efficient Continual Learning Paradigm for Fine-tuning Large Language Models with Instructions
by: Wang, Yifan, et al.
Published: (2024)

LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios
by: Wu, Xiaodong, et al.
Published: (2024)

Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning
by: Tan, Zhaoxuan, et al.
Published: (2024)

Sparse Matrix in Large Language Model Fine-tuning
by: He, Haoze, et al.
Published: (2024)

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data
by: Xie, Juncheng, et al.
Published: (2024)

PILLOW: Enhancing Efficient Instruction Fine-tuning via Prompt Matching
by: Qi, Zhenting, et al.
Published: (2023)

Instruction-Following Pruning for Large Language Models
by: Hou, Bairu, et al.
Published: (2025)

XIFBench: Evaluating Large Language Models on Multilingual Instruction Following
by: Li, Zhenyu, et al.
Published: (2025)

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
by: Zhang, Jingfan, et al.
Published: (2024)

Parrot: Enhancing Multi-Turn Instruction Following for Large Language Models
by: Sun, Yuchong, et al.
Published: (2023)

Evaluating Large Language Models at Evaluating Instruction Following
by: Zeng, Zhiyuan, et al.
Published: (2023)

Constraint Back-translation Improves Complex Instruction Following of Large Language Models
by: Qi, Yunjia, et al.
Published: (2024)

InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification
by: Hu, Yujia, et al.
Published: (2024)

FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models
by: Jiang, Yuxin, et al.
Published: (2023)

IHEval: Evaluating Language Models on Following the Instruction Hierarchy
by: Zhang, Zhihan, et al.
Published: (2025)

In-Context Watermarks for Large Language Models
by: Liu, Yepeng, et al.
Published: (2025)

Mixture-of-Instructions: Aligning Large Language Models via Mixture Prompting
by: Xu, Bowen, et al.
Published: (2024)

When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models
by: Zheng, Yingming, et al.
Published: (2025)

An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
by: Luo, Yun, et al.
Published: (2023)

CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following
by: Zhang, Kaiyan, et al.
Published: (2024)

Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models
by: Sun, Haoran, et al.
Published: (2024)

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition
by: Dong, Guanting, et al.
Published: (2023)

Do we Really Need Visual Instructions? Towards Visual Instruction-Free Fine-tuning for Large Vision-Language Models
by: Liu, Zikang, et al.
Published: (2025)

EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning
by: Huang, Dong, et al.
Published: (2024)

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources
by: Bai, Jiamu, et al.
Published: (2024)

AI Generated Text Detection Using Instruction Fine-tuned Large Language and Transformer-Based Models
by: Guggilla, Chinnappa, et al.
Published: (2025)

Attribute Controlled Fine-tuning for Large Language Models: A Case Study on Detoxification
by: Meng, Tao, et al.
Published: (2024)

Cendol: Open Instruction-tuned Generative Large Language Models for Indonesian Languages
by: Cahyawijaya, Samuel, et al.
Published: (2024)

JudgeLM: Fine-tuned Large Language Models are Scalable Judges
by: Zhu, Lianghui, et al.
Published: (2023)

LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
by: Chen, Yukang, et al.
Published: (2023)