Cover Image

Improving LLM Safety Alignment with Dual-Objective Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Xuandong, Cai, Will, Shi, Tianneng, Huang, David, Lin, Licong, Mei, Song, Song, Dawn
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Cryptography and Security Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs
by: Cai, Will, et al.
Published: (2025)

The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
by: Xiong, Alexander, et al.
Published: (2025)

Self-Sovereign Agent
by: Qu, Wenjie, et al.
Published: (2026)

Cross-Modal Safety Alignment: Is textual unlearning all you need?
by: Chakraborty, Trishna, et al.
Published: (2024)

Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption
by: Liu, Yepeng, et al.
Published: (2025)

Dataset Protection via Watermarked Canaries in Retrieval-Augmented LLMs
by: Liu, Yepeng, et al.
Published: (2025)

Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets
by: Hsiung, Lei, et al.
Published: (2025)

Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs
by: Zhao, Xuandong, et al.
Published: (2024)

Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy
by: Wu, Tong, et al.
Published: (2024)

An Undetectable Watermark for Generative Image Models
by: Gunn, Sam, et al.
Published: (2024)

CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
by: Wang, Zhun, et al.
Published: (2025)

Lifelong Safety Alignment for Language Models
by: Wang, Haoyu, et al.
Published: (2025)

Improved Techniques for Optimization-Based Jailbreaking on Large Language Models
by: Jia, Xiaojun, et al.
Published: (2024)

SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning
by: Zhou, Kaiwen, et al.
Published: (2025)

Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents
by: Li, Xu, et al.
Published: (2026)

RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
by: Yuan, Zhuowen, et al.
Published: (2024)

Measuring Real-World Prompt Injection Attacks in LLM-based Resume Screening
by: Zhang, Mohan, et al.
Published: (2026)

Safety Alignment Can Be Not Superficial With Explicit Safety Signals
by: Li, Jianwei, et al.
Published: (2025)

LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage
by: Nie, Yuzhou, et al.
Published: (2024)

AdvPrefix: An Objective for Nuanced LLM Jailbreaks
by: Zhu, Sicheng, et al.
Published: (2024)

Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment
by: Rad, Melissa Kazemi, et al.
Published: (2025)

Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM
by: Cao, Bochuan, et al.
Published: (2023)

Adversarial Decoding: Generating Readable Documents for Adversarial Objectives
by: Zhang, Collin, et al.
Published: (2024)

Are My Optimized Prompts Compromised? Exploring Vulnerabilities of LLM-based Optimizers
by: Zhao, Andrew, et al.
Published: (2025)

Superficial Safety Alignment Hypothesis
by: Li, Jianwei, et al.
Published: (2024)

Beyond Indistinguishability: Measuring Extraction Risk in LLM APIs
by: Liu, Ruixuan, et al.
Published: (2026)

Certifying LLM Safety against Adversarial Prompting
by: Kumar, Aounon, et al.
Published: (2023)

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
by: Xie, Xuan, et al.
Published: (2024)

On Calibration of LLM-based Guard Models for Reliable Content Moderation
by: Liu, Hongfu, et al.
Published: (2024)

Representation Bending for Large Language Model Safety
by: Yousefpour, Ashkan, et al.
Published: (2025)

Is poisoning a real threat to LLM alignment? Maybe more so than you think
by: Pathmanathan, Pankayaraj, et al.
Published: (2024)

Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness
by: Li, Tung-Ling, et al.
Published: (2025)

A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
by: Wang, Kun, et al.
Published: (2025)

LiSA: Lifelong Safety Adaptation via Conservative Policy Induction
by: Kim, Minbeom, et al.
Published: (2026)

Auto-Tuning Safety Guardrails for Black-Box Large Language Models
by: Abdulkadir, Perry
Published: (2025)

IDT: Dual-Task Adversarial Attacks for Privacy Protection
by: Faustini, Pedro, et al.
Published: (2024)

Improving Privacy Benefits of Redaction
by: Gusain, Vaibhav, et al.
Published: (2025)

In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement
by: Shetty, Anudeex, et al.
Published: (2026)

LLM Unlearning Should Be Form-Independent
by: Ye, Xiaotian, et al.
Published: (2025)

GCG Attack On A Diffusion LLM
by: Neyroud, Ruben, et al.
Published: (2025)