Cover Image

FocuSFT: Bilevel Optimization for Dilution-Aware Long-Context Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pei, Zehua, Zhen, Hui-Ling, Yu, Xianzhi, Pan, Sinno Jialin, Yuan, Mingxuan, Yu, Bei
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
by: Pei, Zehua, et al.
Published: (2024)

Verilog-Evolve: Feedback-Driven and Skill-Evolving Verilog Generation
by: Pei, Zehua, et al.
Published: (2026)

MemDLM: Memory-Enhanced DLM Training
by: Pei, Zehua, et al.
Published: (2026)

Analytical FFN-to-MoE Restructuring via Activation Pattern Analysis
by: Pei, Zehua, et al.
Published: (2025)

BetterV: Controlled Verilog Generation with Discriminative Guidance
by: Pei, Zehua, et al.
Published: (2024)

PreMoE: Proactive Inference for Efficient Mixture-of-Experts
by: Pei, Zehua, et al.
Published: (2025)

Behavioral Fingerprinting of Large Language Models
by: Pei, Zehua, et al.
Published: (2025)

SCOPE: Prompt Evolution for Enhancing Agent Effectiveness
by: Pei, Zehua, et al.
Published: (2025)

Beyond Speedup -- Utilizing KV Cache for Sampling and Reasoning
by: Xing, Zeyu, et al.
Published: (2026)

DiLA: Enhancing LLM Tool Learning with Differential Logic Layer
by: Zhang, Yu, et al.
Published: (2024)

KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inference
by: Li, Xing, et al.
Published: (2025)

Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
by: Long, Quanyu, et al.
Published: (2024)

Large Language Models Know What Makes Exemplary Contexts
by: Long, Quanyu, et al.
Published: (2024)

What Matters For Safety Alignment?
by: Li, Xing, et al.
Published: (2026)

MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
by: Jiang, Weisen, et al.
Published: (2026)

SVDq: 1.25-bit and 410x Key Cache Compression for LLM Attention
by: Yankun, Hong, et al.
Published: (2025)

MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
by: Jiang, Weisen, et al.
Published: (2025)

Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence
by: Sun, Shengyin, et al.
Published: (2026)

SwiftMem: Fast Agentic Memory via Query-aware Indexing
by: Tian, Anxin, et al.
Published: (2026)

SED-SFT: Selectively Encouraging Diversity in Supervised Fine-Tuning
by: Chen, Yijie, et al.
Published: (2026)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
by: Shin, Haebin, et al.
Published: (2025)

Reinforcing Compositional Retrieval: Retrieving Step-by-Step for Composing Informative Contexts
by: Long, Quanyu, et al.
Published: (2025)

Multilingual Jailbreak Challenges in Large Language Models
by: Deng, Yue, et al.
Published: (2023)

Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
by: Wu, Han, et al.
Published: (2025)

PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
by: Sun, Shengyin, et al.
Published: (2026)

MOSS: Efficient and Accurate FP8 LLM Training with Microscaling and Automatic Scaling
by: Zhang, Yu, et al.
Published: (2025)

SFT Doesn't Always Hurt General Capabilities: Revisiting Domain-Specific Fine-Tuning in LLMs
by: Lin, Jiacheng, et al.
Published: (2025)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
by: Hong, Joey, et al.
Published: (2024)

TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
by: Lin, Weizhe, et al.
Published: (2025)

Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggers
by: Long, Quanyu, et al.
Published: (2024)

SFT-TA: Supervised Fine-Tuned Agents in Multi-Agent LLMs for Automated Inductive Thematic Analysis
by: Yi, Seungjun, et al.
Published: (2025)

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
by: Sun, Shengyin, et al.
Published: (2025)

FisherSFT: Data-Efficient Supervised Fine-Tuning of Language Models Using Information Gain
by: Deb, Rohan, et al.
Published: (2025)

One-Token Rollout: Guiding Supervised Fine-Tuning of LLMs with Policy Gradient
by: Ming, Rui, et al.
Published: (2025)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2024)

Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Models
by: Molfese, Francesco Maria, et al.
Published: (2026)

Causal Fine-Tuning under Latent Confounded Shift
by: Yu, Jialin, et al.
Published: (2024)

RIFT: Repurposing Negative Samples via Reward-Informed Fine-Tuning
by: Liu, Zehua, et al.
Published: (2026)

UFT: Unifying Fine-Tuning of SFT and RLHF/DPO/UNA through a Generalized Implicit Reward Function
by: Wang, Zhichao, et al.
Published: (2024)

How to Fine-Tune a Reasoning Model? A Teacher-Student Cooperation Framework to Synthesize Student-Consistent SFT Data
by: Huang, Zixian, et al.
Published: (2026)