Cover Image

Weighted-Reward Preference Optimization for Implicit Model Fusion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Ziyi, Wan, Fanqi, Zhong, Longguang, Shi, Tianyuan, Quan, Xiaojun
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion
by: Zhong, Longguang, et al.
Published: (2025)

FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
by: Yang, Ziyi, et al.
Published: (2025)

FuseChat: Knowledge Fusion of Chat Models
by: Wan, Fanqi, et al.
Published: (2024)

Knowledge Fusion of Chat LLMs: A Preliminary Technical Report
by: Wan, Fanqi, et al.
Published: (2024)

BlockPruner: Fine-grained Pruning for Large Language Models
by: Zhong, Longguang, et al.
Published: (2024)

ThinkSwitcher: When to Think Hard, When to Think Fast
by: Liang, Guosheng, et al.
Published: (2025)

Mutual-Taught for Co-adapting Policy and Reward Models
by: Shi, Tianyuan, et al.
Published: (2025)

Knowledge Fusion of Large Language Models
by: Wan, Fanqi, et al.
Published: (2024)

Advantage-Guided Distillation for Preference Alignment in Small Language Models
by: Gao, Shiping, et al.
Published: (2025)

ProFuser: Progressive Fusion of Large Language Models
by: Shi, Tianyuan, et al.
Published: (2024)

Self-Evolution Fine-Tuning for Policy Optimization
by: Chen, Ruijun, et al.
Published: (2024)

Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization
by: Gao, Shiping, et al.
Published: (2026)

SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models
by: Yang, Ziyi, et al.
Published: (2025)

ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
by: Liu, Tianjian, et al.
Published: (2025)

Knowledge Verification to Nip Hallucination in the Bud
by: Wan, Fanqi, et al.
Published: (2024)

Lookahead Routing for Large Language Models
by: Huang, Canbin, et al.
Published: (2025)

Discriminative Policy Optimization for Token-Level Reward Models
by: Chen, Hongzhan, et al.
Published: (2025)

On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization
by: Lin, Yong, et al.
Published: (2024)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment
by: Wang, Hao, et al.
Published: (2026)

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment
by: Yang, Wen, et al.
Published: (2025)

ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictions
by: Zhang, Xu, et al.
Published: (2024)

RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
by: Wang, Chenglong, et al.
Published: (2024)

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Robust Preference Optimization through Reward Model Distillation
by: Fisch, Adam, et al.
Published: (2024)

RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
by: Yan, Shi-Qi, et al.
Published: (2025)

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
by: Wan, Fanqi, et al.
Published: (2025)

Cool-Fusion: Fuse Large Language Models without Training
by: Liu, Cong, et al.
Published: (2024)

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
by: Zhu, Wenhong, et al.
Published: (2024)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
by: Qi, Xuan, et al.
Published: (2025)

CE-RM: A Pointwise Generative Reward Model Optimized via Two-Stage Rollout and Unified Criteria
by: Hu, Xinyu, et al.
Published: (2026)

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization
by: Sun, Huashan, et al.
Published: (2025)

VRM: Teaching Reward Models to Understand Authentic Human Preferences
by: Liu, Biao, et al.
Published: (2026)

DMoERM: Recipes of Mixture-of-Experts for Effective Reward Modeling
by: Quan, Shanghaoran
Published: (2024)

DSGram: Dynamic Weighting Sub-Metrics for Grammatical Error Correction in the Era of Large Language Models
by: Xie, Jinxiang, et al.
Published: (2024)

Bootstrapping Language Models with DPO Implicit Rewards
by: Chen, Changyu, et al.
Published: (2024)

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
by: Lu, Yining, et al.
Published: (2025)

Group Robust Preference Optimization in Reward-free RLHF
by: Ramesh, Shyam Sundhar, et al.
Published: (2024)

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation
by: Cui, Guofeng, et al.
Published: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
by: Ye, Ziyi, et al.
Published: (2024)