Cover Image

AMoPO: Adaptive Multi-objective Preference Optimization without Reward Models and Reference Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Qi, Ruan, Jingqing, Li, Hao, Zhao, Haodong, Wang, Desheng, Chen, Jiansong, Guanglu, Wan, Cai, Xunliang, Zheng, Zhi, Xu, Tong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

PaTaRM: Bridging Pairwise and Pointwise Signals via Preference-Aware Task-Adaptive Reward Modeling
by: Jian, Ai, et al.
Published: (2025)

When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning
by: Zhang, Xiaoyun, et al.
Published: (2025)

Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy
by: Zhang, Xiaoyun, et al.
Published: (2025)

SimPO: Simple Preference Optimization with a Reference-Free Reward
by: Meng, Yu, et al.
Published: (2024)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
by: Zhu, Zining, et al.
Published: (2025)

ORPO: Monolithic Preference Optimization without Reference Model
by: Hong, Jiwoo, et al.
Published: (2024)

SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
by: Zheng, Binbin, et al.
Published: (2026)

Explicit Preference Optimization: No Need for an Implicit Reward Model
by: Hu, Xiangkun, et al.
Published: (2025)

Weighted-Reward Preference Optimization for Implicit Model Fusion
by: Yang, Ziyi, et al.
Published: (2024)

TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas
by: Jian, Ai, et al.
Published: (2026)

Harmonizing Dense and Sparse Signals in Multi-turn RL: Dual-Horizon Credit Assignment for Industrial Sales Agents
by: Yang, Haojin, et al.
Published: (2026)

AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment
by: Deng, Ruibo, et al.
Published: (2025)

Length-Controlled Margin-Based Preference Optimization without Reference Model
by: Li, Gengxu, et al.
Published: (2025)

Margin-aware Preference Optimization for Aligning Diffusion Models without Reference
by: Hong, Jiwoo, et al.
Published: (2024)

Learning Causal Dynamics Models in Object-Oriented Environments
by: Yu, Zhongwei, et al.
Published: (2024)

Explainable Reinforcement Learning via a Causal World Model
by: Yu, Zhongwei, et al.
Published: (2023)

Multi-Objective and Mixed-Reward Reinforcement Learning via Reward-Decorrelated Policy Optimization
by: Bai, Yang, et al.
Published: (2026)

Multi-Reference Preference Optimization for Large Language Models
by: Le, Hung, et al.
Published: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
by: Liu, Tong, et al.
Published: (2025)

APLOT: Robust Reward Modeling via Adaptive Preference Learning with Optimal Transport
by: Li, Zhuo, et al.
Published: (2025)

Background study of the AMoRE-pilot experiment
by: Agrawal, A., et al.
Published: (2024)

Projected background and sensitivity of AMoRE-II
by: Agrawal, A., et al.
Published: (2024)

Margin Adaptive DPO: Leveraging Reward Model for Granular Control in Preference Optimization
by: Rho, Hyung Gyu
Published: (2025)

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs
by: Kachroo, Darsh, et al.
Published: (2026)

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts
by: Wang, Haoxiang, et al.
Published: (2024)

Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data
by: Dong, Fengxian, et al.
Published: (2026)

LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Space
by: Xin, Detai, et al.
Published: (2026)

Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model
by: Gou, Qi, et al.
Published: (2024)

Mars-PO: Multi-Agent Reasoning System Preference Optimization
by: Lou, Xiaoxuan, et al.
Published: (2024)

In-Context Reward Adaptation for Robust Preference Modeling
by: Sun, Zhenyu, et al.
Published: (2026)

SynPO: Synergizing Descriptiveness and Preference Optimization for Video Detailed Captioning
by: Dang, Jisheng, et al.
Published: (2025)

Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systems
by: Peng, Hao, et al.
Published: (2025)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

Robust Preference Optimization through Reward Model Distillation
by: Fisch, Adam, et al.
Published: (2024)

Visual Preference Optimization with Rubric Rewards
by: Yu, Ya-Qi, et al.
Published: (2026)

Preference Optimization via Contrastive Divergence: Your Reward Model is Secretly an NLL Estimator
by: Chen, Zhuotong, et al.
Published: (2025)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
by: Guo, Siqi, et al.
Published: (2025)

Optimal Transport for LLM Reward Modeling from Noisy Preference
by: Pan, Licheng, et al.
Published: (2026)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)