Cover Image

FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhong, Longguang, Wan, Fanqi, Yang, Ziyi, Liang, Guosheng, Shi, Tianyuan, Quan, Xiaojun
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
by: Yang, Ziyi, et al.
Published: (2025)

Weighted-Reward Preference Optimization for Implicit Model Fusion
by: Yang, Ziyi, et al.
Published: (2024)

FuseChat: Knowledge Fusion of Chat Models
by: Wan, Fanqi, et al.
Published: (2024)

ThinkSwitcher: When to Think Hard, When to Think Fast
by: Liang, Guosheng, et al.
Published: (2025)

Knowledge Fusion of Chat LLMs: A Preliminary Technical Report
by: Wan, Fanqi, et al.
Published: (2024)

BlockPruner: Fine-grained Pruning for Large Language Models
by: Zhong, Longguang, et al.
Published: (2024)

Knowledge Fusion of Large Language Models
by: Wan, Fanqi, et al.
Published: (2024)

Advantage-Guided Distillation for Preference Alignment in Small Language Models
by: Gao, Shiping, et al.
Published: (2025)

ProFuser: Progressive Fusion of Large Language Models
by: Shi, Tianyuan, et al.
Published: (2024)

Self-Evolution Fine-Tuning for Policy Optimization
by: Chen, Ruijun, et al.
Published: (2024)

Mutual-Taught for Co-adapting Policy and Reward Models
by: Shi, Tianyuan, et al.
Published: (2025)

Cool-Fusion: Fuse Large Language Models without Training
by: Liu, Cong, et al.
Published: (2024)

SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Models
by: Yang, Ziyi, et al.
Published: (2025)

ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
by: Liu, Tianjian, et al.
Published: (2025)

Knowledge Verification to Nip Hallucination in the Bud
by: Wan, Fanqi, et al.
Published: (2024)

Lookahead Routing for Large Language Models
by: Huang, Canbin, et al.
Published: (2025)

FuseGen: PLM Fusion for Data-generation based Zero-shot Learning
by: Zou, Tianyuan, et al.
Published: (2024)

ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference Contradictions
by: Zhang, Xu, et al.
Published: (2024)

RPO: Retrieval Preference Optimization for Robust Retrieval-Augmented Generation
by: Yan, Shi-Qi, et al.
Published: (2025)

QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
by: Wan, Fanqi, et al.
Published: (2025)

SoLoPO: Unlocking Long-Context Capabilities in LLMs via Short-to-Long Preference Optimization
by: Sun, Huashan, et al.
Published: (2025)

InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Models
by: Gu, Yanggan, et al.
Published: (2025)

Discriminative Policy Optimization for Token-Level Reward Models
by: Chen, Hongzhan, et al.
Published: (2025)

Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector
by: Yang, Haihui, et al.
Published: (2024)

Evaluating, Understanding, and Improving Constrained Text Generation for Large Language Models
by: Chen, Xiang, et al.
Published: (2023)

Data Proportion Detection for Optimized Data Management for Large Language Models
by: Liang, Hao, et al.
Published: (2024)

Multi-Reference Preference Optimization for Large Language Models
by: Le, Hung, et al.
Published: (2024)

MetFuse: Figurative Fusion between Metonymy and Metaphor
by: Ghosh, Saptarshi, et al.
Published: (2026)

CFunModel: A "Funny" Language Model Capable of Chinese Humor Generation and Processing
by: Yu, Zhenghan, et al.
Published: (2025)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

Boolean-aware Attention for Dense Retrieval
by: Mai, Quan, et al.
Published: (2025)

No Preference Left Behind: Group Distributional Preference Optimization
by: Yao, Binwei, et al.
Published: (2024)

Unleashing Implicit Rewards: Prefix-Value Learning for Distribution-Level Optimization
by: Gao, Shiping, et al.
Published: (2026)

Quantity Matters: Towards Assessing and Mitigating Number Hallucination in Large Vision-Language Models
by: Zhang, Huixuan, et al.
Published: (2024)

Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection
by: Li, Jiatao, et al.
Published: (2025)

CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
by: Xing, Long, et al.
Published: (2025)

Stabilizing Policy Optimization via Logits Convexity
by: Chen, Hongzhan, et al.
Published: (2026)

ROPO: Robust Preference Optimization for Large Language Models
by: Liang, Xize, et al.
Published: (2024)

Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models
by: Zhang, Huixuan, et al.
Published: (2025)

Optimizing RLHF Training for Large Language Models with Stage Fusion
by: Zhong, Yinmin, et al.
Published: (2024)