Copertina

An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Feng, Yuming, Yang, Christy
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
di: Wang, Bo, et al.
Pubblicazione: (2025)

Improving LLM Safety and Helpfulness using SFT and DPO: A Study on OPT-350M
di: Pant, Piyush
Pubblicazione: (2025)

Aligning Large Language Models with Counterfactual DPO
di: Butcher, Bradley
Pubblicazione: (2024)

Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets
di: Pavlidis, Nikolaos, et al.
Pubblicazione: (2025)

Context-DPO: Aligning Language Models for Context-Faithfulness
di: Bi, Baolong, et al.
Pubblicazione: (2024)

An Empirical Study of the Role of Incompleteness and Ambiguity in Interactions with Large Language Models
di: Naik, Riya, et al.
Pubblicazione: (2025)

CollectiveSFT: Scaling Large Language Models for Chinese Medical Benchmark with Collective Instructions in Healthcare
di: Zhu, Jingwei, et al.
Pubblicazione: (2024)

Evaluating Large Language Models on Financial Report Summarization: An Empirical Study
di: Yang, Xinqi, et al.
Pubblicazione: (2024)

Cat-DPO: Category-Adaptive Safety Alignment
di: Yang, Tiankai, et al.
Pubblicazione: (2026)

Small Language Models for Application Interactions: A Case Study
di: Li, Beibin, et al.
Pubblicazione: (2024)

Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective
di: Feng, Duanyu, et al.
Pubblicazione: (2024)

An Empirical Study on Prompt Compression for Large Language Models
di: Zhang, Zheng, et al.
Pubblicazione: (2025)

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning
di: Hong, Joey, et al.
Pubblicazione: (2024)

A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity
di: Lee, Andrew, et al.
Pubblicazione: (2024)

An Empirical Study of Many-to-Many Summarization with Large Language Models
di: Wang, Jiaan, et al.
Pubblicazione: (2025)

daDPO: Distribution-Aware DPO for Distilling Conversational Abilities
di: Zhang, Zhengze, et al.
Pubblicazione: (2025)

Small Language Model as Data Prospector for Large Language Model
di: Ni, Shiwen, et al.
Pubblicazione: (2024)

InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
di: Xie, Congkai, et al.
Pubblicazione: (2025)

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection
di: Liu, Tao, et al.
Pubblicazione: (2026)

Evaluating Large Language Models for Abstract Evaluation Tasks: An Empirical Study
di: Liu, Yinuo, et al.
Pubblicazione: (2026)

Once Upon a Time: Interactive Learning for Storytelling with Small Language Models
di: Martins, Jonas Mayer, et al.
Pubblicazione: (2025)

Efficient and Accurate Memorable Conversation Model using DPO based on sLLM
di: Seo, Youngkyung, et al.
Pubblicazione: (2024)

Quagmires in SFT-RL Post-Training: When High SFT Scores Mislead and What to Use Instead
di: Kang, Feiyang, et al.
Pubblicazione: (2025)

SP^2DPO: An LLM-assisted Semantic Per-Pair DPO Generalization
di: He, Chaoyue, et al.
Pubblicazione: (2026)

An Empirical Analysis of Uncertainty in Large Language Model Evaluations
di: Xie, Qiujie, et al.
Pubblicazione: (2025)

The Dunning-Kruger Effect in Large Language Models: An Empirical Study of Confidence Calibration
di: Ghosh, Sudipta, et al.
Pubblicazione: (2026)

An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference
di: Yamaguchi, Atsuki, et al.
Pubblicazione: (2024)

Unveiling the Learning Mind of Language Models: A Cognitive Framework and Empirical Study
di: Hu, Zhengyu, et al.
Pubblicazione: (2025)

Investigating Persuasion Techniques in Arabic: An Empirical Study Leveraging Large Language Models
di: Alzahrani, Abdurahmman, et al.
Pubblicazione: (2024)

Rethinking DPO: The Role of Rejected Responses in Preference Misalignment
di: Cho, Jay Hyeon, et al.
Pubblicazione: (2025)

Achieving Sparse Activation in Small Language Models
di: Song, Jifeng, et al.
Pubblicazione: (2024)

MixDPO: Modeling Preference Strength for Pluralistic Alignment
di: Imai, Saki, et al.
Pubblicazione: (2026)

Data Generation Using Large Language Models for Text Classification: An Empirical Case Study
di: Li, Yinheng, et al.
Pubblicazione: (2024)

An Empirical Study on Large Language Models in Accuracy and Robustness under Chinese Industrial Scenarios
di: Li, Zongjie, et al.
Pubblicazione: (2024)

Teaching with Lies: Curriculum DPO on Synthetic Negatives for Hallucination Detection
di: Pandit, Shrey, et al.
Pubblicazione: (2025)

sDPO: Don't Use Your Data All at Once
di: Kim, Dahyun, et al.
Pubblicazione: (2024)

DPO Meets PPO: Reinforced Token Optimization for RLHF
di: Zhong, Han, et al.
Pubblicazione: (2024)

Parallax: Parameterized Local Linear Attention for Language Modeling
di: Zuo, Yifei, et al.
Pubblicazione: (2026)

UrbanPlanBench: A Comprehensive Urban Planning Benchmark for Evaluating Large Language Models
di: Zheng, Yu, et al.
Pubblicazione: (2025)

Small Language Models for Emergency Departments Decision Support: A Benchmark Study
di: Wang, Zirui, et al.
Pubblicazione: (2025)