Image de couverture de livre

Minor SFT loss for LLM fine-tune to increase performance and reduce model deviation

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Xie, Shiming, Chen, Hong, Yu, Fred, Sun, Zeye, Wu, Xiuyu
Format:	Preprint
Publié:	2024
Sujets:	Artificial Intelligence Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Minor DPO reject penalty to increase training robustness
par: Xie, Shiming, et autres
Publié: (2024)

How does fine-tuning improve sensorimotor representations in large language models?
par: Wu, Minghua, et autres
Publié: (2026)

PDC & DM-SFT: A Road for LLM SQL Bug-Fix Enhancing
par: Duan, Yiwen, et autres
Publié: (2024)

Reinforcement learning fine-tuning of language model for instruction following and math reasoning
par: Han, Yifu, et autres
Publié: (2025)

Evolution of meta's llama models and parameter-efficient fine-tuning of large language models: a survey
par: Abdullah, Abdulhady Abas, et autres
Publié: (2025)

Order Doesn't Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation
par: He, Qianxi, et autres
Publié: (2025)

Retention analysis of edited knowledge after fine-tuning
par: Wen, Fufang, et autres
Publié: (2025)

TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
par: Ma, Zerun, et autres
Publié: (2026)

Rethinking harmless refusals when fine-tuning foundation models
par: Pop, Florin, et autres
Publié: (2024)

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
par: Liu, Tao, et autres
Publié: (2026)

SFT-then-RL Outperforms Mixed-Policy Methods for LLM Reasoning
par: Limozin, Alexis, et autres
Publié: (2026)

Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation
par: Wang, Yibo, et autres
Publié: (2025)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
par: Kang, Feiyang, et autres
Publié: (2025)

Metaphor identification using large language models: A comparison of RAG, prompt engineering, and fine-tuning
par: Fuoli, Matteo, et autres
Publié: (2025)

Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following
par: Zeng, Jie, et autres
Publié: (2025)

Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following
par: Ren, Qingyu, et autres
Publié: (2025)

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models
par: Ren, Qingyu, et autres
Publié: (2025)

Agent Data Protocol: Unifying Datasets for Diverse, Effective Fine-tuning of LLM Agents
par: Song, Yueqi, et autres
Publié: (2025)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
par: Du, Yuhao, et autres
Publié: (2025)

Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems
par: Lakatos, Robert, et autres
Publié: (2024)

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
par: Shin, Haebin, et autres
Publié: (2025)

Continual SFT Matches Multimodal RLHF with Negative Supervision
par: Zhu, Ke, et autres
Publié: (2024)

OpenMedLM: Prompt engineering can out-perform fine-tuning in medical question-answering with open-source large language models
par: Maharjan, Jenish, et autres
Publié: (2024)

Using Optimal Transport as Alignment Objective for fine-tuning Multilingual Contextualized Embeddings
par: Alqahtani, Sawsan, et autres
Publié: (2021)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
par: Hong, Joey, et autres
Publié: (2024)

Improving embedding with contrastive fine-tuning on small datasets with expert-augmented scores
par: Lu, Jun, et autres
Publié: (2024)

The impact of fine tuning in LLaMA on hallucinations for named entity extraction in legal documentation
par: Vargas, Francisco, et autres
Publié: (2025)

FineTuneBench: How well do commercial fine-tuning APIs infuse knowledge into LLMs?
par: Wu, Eric, et autres
Publié: (2024)

Response-free item difficulty modelling for multiple-choice items with fine-tuned transformers: Component-wise representation and multi-task learning
par: Netík, Jan, et autres
Publié: (2026)

An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models
par: Feng, Yuming, et autres
Publié: (2026)

RandLoRA: Full-rank parameter-efficient fine-tuning of large models
par: Albert, Paul, et autres
Publié: (2025)

Airavata: Introducing Hindi Instruction-tuned LLM
par: Gala, Jay, et autres
Publié: (2024)

Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M
par: Pant, Piyush
Publié: (2025)

LoX: Low-Rank Extrapolation Robustifies LLM Safety Against Fine-tuning
par: Perin, Gabriel J., et autres
Publié: (2025)

A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks
par: Zhang, Tianyi, et autres
Publié: (2024)

Decoupling KL and Trajectories: A Unified Perspective for SFT, DAgger, Offline RL, and OPD in LLM Distillation
par: Zhao, Anhao, et autres
Publié: (2026)

Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
par: Wang, Sudong, et autres
Publié: (2026)

RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning
par: Morandi, Andrea
Publié: (2026)

SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection
par: Shen, Han, et autres
Publié: (2024)

Uncertainty quantification in fine-tuned LLMs using LoRA ensembles
par: Balabanov, Oleksandr, et autres
Publié: (2024)