Imagen de Portada

SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Lou, Xingzhou, Zhang, Junge, Xie, Jian, Liu, Lifeng, Yan, Dong, Huang, Kaiqi
Formato:	Preprint
Publicado:	2024
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown
por: Lou, Xingzhou, et al.
Publicado: (2024)

Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Models
por: Lou, Xingzhou, et al.
Publicado: (2024)

Reward-Robust RLHF in LLMs
por: Yan, Yuzi, et al.
Publicado: (2024)

Calibration-Aware Policy Optimization for Reasoning LLMs
por: Wang, Ziqi, et al.
Publicado: (2026)

SPO: Sequential Monte Carlo Policy Optimisation
por: Macfarlane, Matthew V, et al.
Publicado: (2024)

Explicit Preference Optimization: No Need for an Implicit Reward Model
por: Hu, Xiangkun, et al.
Publicado: (2025)

InSPO: Unlocking Intrinsic Self-Reflection for LLM Preference Optimization
por: Li, Yu, et al.
Publicado: (2025)

TAPE: Leveraging Agent Topology for Cooperative Multi-Agent Policy Gradient
por: Lou, Xingzhou, et al.
Publicado: (2023)

PREFINE: Preference-Based Implicit Reward and Cost Fine-Tuning for Safety Alignment
por: Verma, Richa, et al.
Publicado: (2026)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
por: Lin, Yong, et al.
Publicado: (2024)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
por: Yang, Rui, et al.
Publicado: (2024)

Reward-Augmented Data Enhances Direct Preference Alignment of LLMs
por: Zhang, Shenao, et al.
Publicado: (2024)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
por: Wang, Haoxiang, et al.
Publicado: (2024)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
por: Sun, Shengyang, et al.
Publicado: (2025)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
por: Wang, Haoxiang, et al.
Publicado: (2024)

PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
por: Lin, Baijiong, et al.
Publicado: (2025)

Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modeling
por: Xie, Lipeng, et al.
Publicado: (2025)

Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards
por: Chen, Honghao, et al.
Publicado: (2025)

From Demonstrations to Rewards: Alignment Without Explicit Human Preferences
por: Zeng, Siliang, et al.
Publicado: (2025)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
por: Huang, Kexin, et al.
Publicado: (2025)

CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation
por: He, Han, et al.
Publicado: (2024)

Property-driven Protein Inverse Folding With Multi-Objective Preference Alignment
por: Hou, Xiaoyang, et al.
Publicado: (2026)

Bootstrapping Language Models with DPO Implicit Rewards
por: Chen, Changyu, et al.
Publicado: (2024)

Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
por: Wang, Austin, et al.
Publicado: (2026)

PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation
por: Liu, Runze, et al.
Publicado: (2023)

SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins
por: Ko, Jongwoo, et al.
Publicado: (2024)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
por: Qi, Xuan, et al.
Publicado: (2025)

IRPM: Intergroup Relative Preference Modeling for Pointwise Generative Reward Models
por: Song, Haonan, et al.
Publicado: (2026)

Robust Reward Alignment via Hypothesis Space Batch Cutting
por: Xie, Zhixian, et al.
Publicado: (2025)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
por: Jiang, Zaifan, et al.
Publicado: (2023)

Regularized Conditional Diffusion Model for Multi-Task Preference Alignment
por: Yu, Xudong, et al.
Publicado: (2024)

Multi-Objective Reward and Preference Optimization: Theory and Algorithms
por: Agnihotri, Akhil
Publicado: (2025)

Similarity as Reward Alignment: Robust and Versatile Preference-based Reinforcement Learning
por: Rajaram, Sara, et al.
Publicado: (2025)

Transfer in Sequential Multi-armed Bandits via Reward Samples
por: R, Rahul N, et al.
Publicado: (2024)

Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning
por: Zhang, Heng, et al.
Publicado: (2026)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
por: Ye, Ziyi, et al.
Publicado: (2024)

Bayesian Reward Models for LLM Alignment
por: Yang, Adam X., et al.
Publicado: (2024)

Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment
por: Wang, Chaoqi, et al.
Publicado: (2025)

Multi-Objective Preference Optimization: Improving Human Alignment of Generative Models
por: Agnihotri, Akhil, et al.
Publicado: (2025)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
por: Gu, Yanggan, et al.
Publicado: (2025)