Cover Image

Objective Matters: Fine-Tuning Objectives Shape Safety, Robustness, and Persona Drift

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vennemeyer, Daniel, Pandey, Punya Syon, Duong, Phan Anh, Umeokoli, Michael, Ratnam, Samuel
Format:	Preprint
Published:	2026
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Accidental Vulnerability: Factors in Fine-Tuning that Shift Model Safeguards
by: Pandey, Punya Syon, et al.
Published: (2025)

BinaryPPO: Efficient Policy Optimization for Binary Classification
by: Pandey, Punya Syon, et al.
Published: (2026)

CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressures
by: Pandey, Punya Syon, et al.
Published: (2025)

SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requests
by: Pandey, Punya Syon, et al.
Published: (2025)

Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs
by: Vennemeyer, Daniel, et al.
Published: (2025)

Drifting Objectives for Refining Discrete Diffusion Language Models
by: Oba, Daisuke, et al.
Published: (2026)

COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework
by: Ren, Yinuo, et al.
Published: (2024)

EMORL: Ensemble Multi-Objective Reinforcement Learning for Efficient and Flexible LLM Fine-Tuning
by: Kong, Lingxiao, et al.
Published: (2025)

Quriosity: Analyzing Human Questioning Behavior and Causal Inquiry through Curiosity-Driven Queries
by: Ceraolo, Roberto, et al.
Published: (2024)

CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
by: Draye, Florent, et al.
Published: (2026)

Uncovering Cross-Objective Interference in Multi-Objective Alignment
by: Lu, Yining, et al.
Published: (2026)

Robust Multi-Objective Preference Alignment with Online DPO
by: Gupta, Raghav, et al.
Published: (2025)

Improving LLM Safety Alignment with Dual-Objective Optimization
by: Zhao, Xuandong, et al.
Published: (2025)

AutoFT: Learning an Objective for Robust Fine-Tuning
by: Choi, Caroline, et al.
Published: (2024)

Data Diversity Matters for Robust Instruction Tuning
by: Bukharin, Alexander, et al.
Published: (2023)

I Can't Believe It's Not Robust: Catastrophic Collapse of Safety Classifiers under Embedding Drift
by: Sahoo, Subramanyam, et al.
Published: (2026)

Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders
by: Harrasse, Abir, et al.
Published: (2025)

Multi-Objective Alignment of Language Models for Personalized Psychotherapy
by: Beikzadeh, Mehrab, et al.
Published: (2026)

Discovering Implicit Large Language Model Alignment Objectives
by: Chen, Edward, et al.
Published: (2026)

Bradley-Terry and Multi-Objective Reward Modeling Are Complementary
by: Zhang, Zhiwei, et al.
Published: (2025)

FTFT: Efficient and Robust Fine-Tuning by Transferring Training Dynamics
by: Du, Yupei, et al.
Published: (2023)

Instruction Fine-Tuning: Does Prompt Loss Matter?
by: Huerta-Enochian, Mathew, et al.
Published: (2024)

What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning
by: Zhu, Yuchang, et al.
Published: (2025)

Benign Samples Matter! Fine-tuning On Outlier Benign Samples Severely Breaks Safety
by: Guan, Zihan, et al.
Published: (2025)

Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
by: Goel, Jyotin, et al.
Published: (2026)

Fine Tuning Methods for Low-resource Languages
by: Bakkenes, Tim, et al.
Published: (2025)

RLP: Reinforcement as a Pretraining Objective
by: Hatamizadeh, Ali, et al.
Published: (2025)

Reward-free Alignment for Conflicting Objectives
by: Chen, Peter, et al.
Published: (2026)

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
by: Lu, Yining, et al.
Published: (2025)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
by: Wang, Haoxiang, et al.
Published: (2024)

The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives
by: Bou, Matthieu, et al.
Published: (2025)

Multi-Objective Large Language Model Unlearning
by: Pan, Zibin, et al.
Published: (2024)

Aligning the Objective of LLM-based Program Repair
by: Xu, Junjielong, et al.
Published: (2024)

Pareto Multi-Objective Alignment for Language Models
by: He, Qiang, et al.
Published: (2025)

Mixture-of-Personas Language Models for Population Simulation
by: Bui, Ngoc, et al.
Published: (2025)

Objective Metrics for Evaluating Large Language Models Using External Data Sources
by: Du, Haoze, et al.
Published: (2025)

OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment
by: Lin, Liang, et al.
Published: (2025)

Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study
by: Ponkshe, Kaustubh, et al.
Published: (2025)

ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning
by: She, Jingyuan Selena, et al.
Published: (2023)

FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data
by: Abdelkadir, Nuredin Ali, et al.
Published: (2026)