Imagen de Portada

Preference Poisoning Attacks on Reward Model Learning

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wu, Junlin, Wang, Jiongxiao, Xiao, Chaowei, Wang, Chenguang, Zhang, Ning, Vorobeychik, Yevgeniy
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning Artificial Intelligence Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

RLHFPoison: Reward Poisoning Attack for Reinforcement Learning with Human Feedback in Large Language Models
por: Wang, Jiongxiao, et al.
Publicado: (2023)

Verified Safe Reinforcement Learning for Neural Network Dynamic Models
por: Wu, Junlin, et al.
Publicado: (2024)

Learning Linear Utility Functions From Pairwise Comparison Queries
por: Ge, Luise, et al.
Publicado: (2024)

Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
por: Ma, Xinhang, et al.
Publicado: (2026)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
por: Huang, Kexin, et al.
Publicado: (2025)

Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data
por: Baumgärtner, Tim, et al.
Publicado: (2024)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
por: Wu, Junkang, et al.
Publicado: (2024)

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
por: Wang, Xinyan, et al.
Publicado: (2026)

AutoRule: Reasoning Chain-of-thought Extracted Rule-based Rewards Improve Preference Learning
por: Wang, Tevin, et al.
Publicado: (2025)

Hidden State Poisoning Attacks against Mamba-based Language Models
por: Mercier, Alexandre Le, et al.
Publicado: (2026)

Self-Generated Critiques Boost Reward Modeling for Language Models
por: Yu, Yue, et al.
Publicado: (2024)

Multi-Agent Reinforcement Learning for Assessing False-Data Injection Attacks on Transportation Networks
por: Eghtesad, Taha, et al.
Publicado: (2023)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
por: Wang, Haoxiang, et al.
Publicado: (2024)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
por: Ye, Ziyi, et al.
Publicado: (2024)

CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planning
por: Wang, Jun, et al.
Publicado: (2025)

West-of-N: Synthetic Preferences for Self-Improving Reward Models
por: Pace, Alizée, et al.
Publicado: (2024)

T-REG: Preference Optimization with Token-Level Reward Regularization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

RewardAnything: Generalizable Principle-Following Reward Models
por: Yu, Zhuohao, et al.
Publicado: (2025)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
por: Liu, Chris Yuhao, et al.
Publicado: (2025)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
por: Rafailov, Rafael, et al.
Publicado: (2023)

Online Feedback Efficient Active Target Discovery in Partially Observable Environments
por: Sarkar, Anindya, et al.
Publicado: (2025)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
por: Zhu, Mingkang, et al.
Publicado: (2025)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
por: Zhu, Zining, et al.
Publicado: (2025)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
por: Yang, Rui, et al.
Publicado: (2024)

RuAG: Learned-rule-augmented Generation for Large Language Models
por: Zhang, Yudi, et al.
Publicado: (2024)

Reward Shaping to Mitigate Reward Hacking in RLHF
por: Fu, Jiayi, et al.
Publicado: (2025)

WPO: Enhancing RLHF with Weighted Preference Optimization
por: Zhou, Wenxuan, et al.
Publicado: (2024)

RM-R1: Reward Modeling as Reasoning
por: Chen, Xiusi, et al.
Publicado: (2025)

Measuring Social Norms of Large Language Models
por: Yuan, Ye, et al.
Publicado: (2024)

Measuring Vision-Language STEM Skills of Neural Models
por: Shen, Jianhao, et al.
Publicado: (2024)

Reinforcement Learning with Conditional Expectation Reward
por: Xiao, Changyi, et al.
Publicado: (2026)

Axioms for AI Alignment from Human Feedback
por: Ge, Luise, et al.
Publicado: (2024)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

More Bang for the Buck: Process Reward Modeling with Entropy-Driven Uncertainty
por: Cao, Lang, et al.
Publicado: (2025)

Preference Learning Algorithms Do Not Learn Preference Rankings
por: Chen, Angelica, et al.
Publicado: (2024)

Learning Interpretable Policies in Hindsight-Observable POMDPs through Partially Supervised Reinforcement Learning
por: Lanier, Michael, et al.
Publicado: (2024)

Active Preference Learning for Large Language Models
por: Muldrew, William, et al.
Publicado: (2024)

Co-Evolution of Policy and Internal Reward for Language Agents
por: Wang, Xinyu, et al.
Publicado: (2026)

Inference-Time Scaling for Generalist Reward Modeling
por: Liu, Zijun, et al.
Publicado: (2025)

Selective Preference Optimization via Token-Level Reward Function Estimation
por: Yang, Kailai, et al.
Publicado: (2024)