Cover Image

Rethinking Supervised Fine-Tuning: Emphasizing Key Answer Tokens for Improved LLM Accuracy

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shi, Xiaofeng, Kou, Qian, Li, Yuduo, Zhou, Hua
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RAFT: Data Refinement and Adaptive Distillation for Domain Fine-Tuning with Alleviated Forgetting
by: Li, Yuduo, et al.
Published: (2026)

Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning
by: Pang, Jinlong, et al.
Published: (2025)

From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning
by: Li, Yafu, et al.
Published: (2025)

Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levels
by: Ye, Junjie, et al.
Published: (2025)

SPAR: Scholar Paper Retrieval with LLM-based Agents for Enhanced Academic Search
by: Shi, Xiaofeng, et al.
Published: (2025)

No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
by: Cencerrado, Iván Vicente Moreno, et al.
Published: (2025)

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
by: Shen, Zhanming, et al.
Published: (2026)

Semantic Density Effect (SDE): Maximizing Information Per Token Improves LLM Accuracy
by: Ahmed, Amr
Published: (2026)

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting
by: Ding, Fei, et al.
Published: (2025)

Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
by: Ban, Hao, et al.
Published: (2025)

A Scaling Law for Token Efficiency in LLM Fine-Tuning Under Fixed Compute Budgets
by: Lagasse, Ryan, et al.
Published: (2025)

Phased Instruction Fine-Tuning for Large Language Models
by: Pang, Wei, et al.
Published: (2024)

Explainable Token-level Noise Filtering for LLM Fine-tuning Datasets
by: Yang, Yuchen, et al.
Published: (2026)

IA2: Alignment with ICL Activations Improves Supervised Fine-Tuning
by: Mishra, Aayush, et al.
Published: (2025)

TokenSeek: Memory Efficient Fine Tuning via Instance-Aware Token Ditching
by: Zeng, Runjia, et al.
Published: (2026)

Stabilizing LLM Supervised Fine-Tuning via Explicit Distributional Control
by: Wang, Xinyu, et al.
Published: (2026)

Proximal Supervised Fine-Tuning
by: Zhu, Wenhong, et al.
Published: (2025)

Rethinking the Bounds of LLM Reasoning: Are Multi-Agent Discussions the Key?
by: Wang, Qineng, et al.
Published: (2024)

Aqulia-Med LLM: Pioneering Full-Process Open-Source Medical Language Models
by: Zhao, Lulu, et al.
Published: (2024)

Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning
by: Shernazarov, Ulugbek, et al.
Published: (2026)

Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning
by: Qian, Livia, et al.
Published: (2026)

TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning
by: Ma, Dabiao, et al.
Published: (2025)

Team-Based Self-Play With Dual Adaptive Weighting for Fine-Tuning LLMs
by: Li, Wu, et al.
Published: (2026)

Supervised Fine-Tuning or In-Context Learning? Evaluating LLMs for Clinical NER
by: Baroian, Andrei
Published: (2025)

Supervised Fine-Tuning as Inverse Reinforcement Learning
by: Sun, Hao
Published: (2024)

Key-Element-Informed sLLM Tuning for Document Summarization
by: Ryu, Sangwon, et al.
Published: (2024)

CareBot: A Pioneering Full-Process Open-Source Medical Language Model
by: Zhao, Lulu, et al.
Published: (2024)

Beyond Perplexity: A Lightweight Benchmark for Knowledge Retention in Supervised Fine-Tuning
by: Shabgahi, Soheil Zibakhsh, et al.
Published: (2026)

Token-level Data Selection for Safe LLM Fine-tuning
by: Li, Yanping, et al.
Published: (2026)

Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process
by: Hua, Ermo, et al.
Published: (2024)

GraphGen: Enhancing Supervised Fine-Tuning for LLMs with Knowledge-Driven Synthetic Data Generation
by: Chen, Zihong, et al.
Published: (2025)

Supervised Fine Tuning of Large Language Models for Domain Specific Knowledge Graph Construction:A Case Study on Hunan's Historical Celebrities
by: Hao, Junjie, et al.
Published: (2025)

Understanding Forgetting in LLM Supervised Fine-Tuning and Preference Learning -- A Convex Optimization Perspective
by: Fernando, Heshan, et al.
Published: (2024)

LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning
by: Mao, Yansheng, et al.
Published: (2024)

Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling
by: Huang, Zeyu, et al.
Published: (2025)

From Answers to Questions: EQGBench for Evaluating LLMs' Educational Question Generation
by: Zhou, Chengliang, et al.
Published: (2025)

Rethinking Table Instruction Tuning
by: Deng, Naihao, et al.
Published: (2025)

Semantic Loss Guided Data Efficient Supervised Fine Tuning for Safe Responses in LLMs
by: Lu, Yuxiao, et al.
Published: (2024)

HeartLLM: Discretized ECG Tokenization for LLM-Based Diagnostic Reasoning
by: Yang, Jinning, et al.
Published: (2025)

Banishing LLM Hallucinations Requires Rethinking Generalization
by: Li, Johnny, et al.
Published: (2024)