Cover Image

Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Yu, Yu, Longxuan, Shahgir, Haz Sameen, Wei, Zhipeng, Liu, Hui, Erichson, N. Benjamin, Dong, Yue
Format:	Preprint
Published:	2026
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Do Reasoning LLMs Refuse What They Infer in Long Contexts?
by: Fu, Yu, et al.
Published: (2026)

Asymmetric Bias in Text-to-Image Generation with Adversarial Attacks
by: Shahgir, Haz Sameen, et al.
Published: (2023)

Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection
by: Wei, Zhipeng, et al.
Published: (2024)

Harnessing the Unseen: The Hidden Influence of Intrinsic Knowledge in Long-Context Language Models
by: Fu, Yu, et al.
Published: (2025)

AbFlowNet: Optimizing Antibody-Antigen Binding Energy via Diffusion-GFlowNet Fusion
by: Abir, Abrar Rahman, et al.
Published: (2025)

Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimization
by: Niu, Yifan, et al.
Published: (2025)

Multilingual Safety Alignment via Self-Distillation
by: Qin, Ruiyang, et al.
Published: (2026)

Block-Biased Mamba for Long-Range Sequence Processing
by: Yu, Annan, et al.
Published: (2025)

Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable
by: Huang, Tiansheng, et al.
Published: (2025)

Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection
by: Sun, Guanglong, et al.
Published: (2026)

ExpertGenQA: Open-ended QA generation in Specialized Domains
by: Shahgir, Haz Sameen, et al.
Published: (2025)

A Real-Time System to Populate FRA Form 57 from News
by: Lim, Chansong, et al.
Published: (2025)

Continuity Laws for Sequential Models
by: Yu, Annan, et al.
Published: (2026)

HOPE for a Robust Parameterization of Long-memory State Space Models
by: Yu, Annan, et al.
Published: (2024)

LLM Safety Alignment is Divergence Estimation in Disguise
by: Haldar, Rajdeep, et al.
Published: (2025)

VLMs Need Words: Vision Language Models Ignore Visual Detail In Favor of Semantic Anchors
by: Shahgir, Haz Sameen, et al.
Published: (2026)

TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment
by: Tan, Zhewen, et al.
Published: (2026)

Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment
by: Zaree, Pedram, et al.
Published: (2025)

The Causal Impact of Tool Affordance on Safety Alignment in LLM Agents
by: Yu, Shasha, et al.
Published: (2026)

Advancing LLM Safe Alignment with Safety Representation Ranking
by: Du, Tianqi, et al.
Published: (2025)

Black-Box Behavioral Distillation Breaks Safety Alignment in Medical LLMs
by: Jahan, Sohely, et al.
Published: (2025)

PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
by: Xu, Yuanda, et al.
Published: (2026)

Mitigating the Alignment Tax of RLHF
by: Lin, Yong, et al.
Published: (2023)

Revisiting Robustness for LLM Safety Alignment via Selective Geometry Control
by: Yang, Yonghui, et al.
Published: (2026)

Guardrails in Logit Space: Safety Token Regularization for LLM Alignment
by: Bach, Thong, et al.
Published: (2026)

Alignment with Preference Optimization Is All You Need for LLM Safety
by: Alami, Reda, et al.
Published: (2024)

Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challenges
by: Lu, Haoran, et al.
Published: (2025)

Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!
by: Zhou, Zhanhui, et al.
Published: (2024)

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation
by: Sang, Hejian, et al.
Published: (2026)

Uncovering Logit Suppression Vulnerabilities in LLM Safety Alignment
by: Li, Yuxi, et al.
Published: (2024)

LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
by: Yang, Junxiao, et al.
Published: (2026)

Sharpen Your Flow: Sharpness-Aware Sampling for Flow Matching
by: Gupta, Aditi, et al.
Published: (2026)

Light Alignment Improves LLM Safety via Model Self-Reflection with a Single Neuron
by: Shen, Sicheng, et al.
Published: (2026)

HydroDiffusion: Diffusion-Based Probabilistic Streamflow Forecasting with a State Space Backbone
by: Wang, Yihan, et al.
Published: (2025)

MACE: A Hybrid LLM Serving System with Colocated SLO-aware Continuous Retraining Alignment
by: Li, Yufei, et al.
Published: (2025)

TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
by: Hui, Zheng, et al.
Published: (2025)

Improving LLM Safety Alignment with Dual-Objective Optimization
by: Zhao, Xuandong, et al.
Published: (2025)

IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models
by: Shahgir, Haz Sameen, et al.
Published: (2024)

On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation
by: Han, Andy, et al.
Published: (2026)

Shape it Up! Restoring LLM Safety during Finetuning
by: Peng, ShengYun, et al.
Published: (2025)