Cover Image

Targeting Misalignment: A Conflict-Aware Framework for Reward-Model-based LLM Alignment

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Zixuan, Khajavi, Siavash H., Jiang, Guangkai, Liu, Xinru
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding
by: Liu, Zixuan, et al.
Published: (2025)

MemBoost: A Memory-Boosted Framework for Cost-Aware LLM Inference
by: Köster, Joris, et al.
Published: (2026)

LLM Benchmark-User Need Misalignment for Climate Change
by: Liu, Oucheng, et al.
Published: (2026)

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
by: Zhang, Hongbin, et al.
Published: (2025)

Reward-free Alignment for Conflicting Objectives
by: Chen, Peter, et al.
Published: (2026)

HAF-RM: A Hybrid Alignment Framework for Reward Model Training
by: Liu, Shujun, et al.
Published: (2024)

Teaching LLM to be Persuasive: Reward-Enhanced Policy Optimization for Alignment from Heterogeneous Rewards
by: Zeng, Xia, et al.
Published: (2025)

RMB: Comprehensively Benchmarking Reward Models in LLM Alignment
by: Zhou, Enyu, et al.
Published: (2024)

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
by: Lu, Yining, et al.
Published: (2025)

OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
by: Liu, Tianci, et al.
Published: (2025)

Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability
by: Shu, Dong, et al.
Published: (2025)

Misaligned by Reward: Socially Undesirable Preferences in LLMs
by: Ghazaryan, Gayane, et al.
Published: (2026)

RGB-Th-Bench: A Dense benchmark for Visual-Thermal Understanding of Vision Language Models
by: Moshtaghi, Mehdi, et al.
Published: (2025)

Synthetic imagery for fuzzy object detection: A comparative study
by: Khajavi, Siavash H., et al.
Published: (2024)

Reinforcement Learning Amplifies Emergent Misalignment from Harmless Rewards
by: Jørgenvåg, Magnus, et al.
Published: (2026)

AlphaPO: Reward Shape Matters for LLM Alignment
by: Gupta, Aman, et al.
Published: (2025)

MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modeling
by: Feng, Zhaopeng, et al.
Published: (2025)

Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
by: Cheng, Ruoxi, et al.
Published: (2025)

A Survey on Progress in LLM Alignment from the Perspective of Reward Design
by: Ji, Miaomiao, et al.
Published: (2025)

RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
by: Yang, Daniel, et al.
Published: (2026)

Detecting Prefix Bias in LLM-based Reward Models
by: Kumar, Ashwin, et al.
Published: (2025)

One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
by: Cai, Hongru, et al.
Published: (2026)

UniARM: Towards a Unified Autoregressive Reward Model for Multi-Objective Test-Time Alignment
by: Xie, Hongyan, et al.
Published: (2026)

An Investigation into Value Misalignment in LLM-Generated Texts for Cultural Heritage
by: Bu, Fan, et al.
Published: (2025)

Hypothesis Testing for Quantifying LLM-Human Misalignment in Multiple Choice Settings
by: Hong, Harbin, et al.
Published: (2025)

Entropy Aware Reward Guidance for Diffusion Language Model Alignment
by: Tejaswi, Atula, et al.
Published: (2026)

SAFENLIDB: A Privacy-Preserving Safety Alignment Framework for LLM-based Natural Language Database Interfaces
by: Liu, Ruiheng, et al.
Published: (2025)

RM-Distiller: Exploiting Generative LLM for Reward Model Distillation
by: Zhou, Hongli, et al.
Published: (2026)

Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignment
by: Liu, Yan, et al.
Published: (2024)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
by: Sun, Shengyang, et al.
Published: (2025)

Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLM
by: Tan, Chenkun, et al.
Published: (2025)

Persona-Aware Alignment Framework for Personalized Dialogue Generation
by: Li, Guanrong, et al.
Published: (2025)

Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models
by: Singla, Somanshu, et al.
Published: (2024)

MRScore: Evaluating Radiology Report Generation with LLM-based Reward System
by: Liu, Yunyi, et al.
Published: (2024)

Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
by: Wu, Tianhao, et al.
Published: (2024)

Hierarchical Contextual Manifold Alignment for Structuring Latent Representations in Large Language Models
by: Dong, Meiquan, et al.
Published: (2025)

GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment
by: Xu, Yuancheng, et al.
Published: (2024)

Reuse Your Rewards: Reward Model Transfer for Zero-Shot Cross-Lingual Alignment
by: Wu, Zhaofeng, et al.
Published: (2024)

Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models
by: Chan, Yik Siu, et al.
Published: (2025)

Normative Conflicts and Shallow AI Alignment
by: Millière, Raphaël
Published: (2025)