Copertina

Larger or Smaller Reward Margins to Select Preferences for Alignment?

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Huang, Kexin, Wu, Junkang, Chen, Ziqian, Wang, Xue, Gao, Jinyang, Ding, Bolin, Wu, Jiancan, He, Xiangnan, Wang, Xiang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Artificial Intelligence Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
di: Wu, Junkang, et al.
Pubblicazione: (2024)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
di: Wu, Junkang, et al.
Pubblicazione: (2024)

RePO: Understanding Preference Learning Through ReLU-Based Optimization
di: Wu, Junkang, et al.
Pubblicazione: (2025)

$β$-DPO: Direct Preference Optimization with Dynamic $β$
di: Wu, Junkang, et al.
Pubblicazione: (2024)

Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
di: Wu, Junkang, et al.
Pubblicazione: (2025)

bi-GRPO: Bidirectional Optimization for Jailbreak Backdoor Injection on LLMs
di: Ji, Wence, et al.
Pubblicazione: (2025)

On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
di: Huang, Kexin, et al.
Pubblicazione: (2026)

Robust Preference Optimization via Dynamic Target Margins
di: Sun, Jie, et al.
Pubblicazione: (2025)

Distill Not Only Data but Also Rewards: Can Smaller Language Models Surpass Larger Ones?
di: Zhang, Yudi, et al.
Pubblicazione: (2025)

Less is More: Improving LLM Alignment via Preference Data Selection
di: Deng, Xun, et al.
Pubblicazione: (2025)

Implicit Cross-Lingual Rewarding for Efficient Multilingual Preference Alignment
di: Yang, Wen, et al.
Pubblicazione: (2025)

AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization
di: Lu, Jinda, et al.
Pubblicazione: (2025)

R^2-Mem: Reflective Experience for Memory Search
di: Wang, Xinyuan, et al.
Pubblicazione: (2026)

On Negative-aware Preference Optimization for Recommendation
di: Ding, Chenlu, et al.
Pubblicazione: (2025)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
di: Xiao, Jianfei, et al.
Pubblicazione: (2026)

Incentivizing Strong Reasoning from Weak Supervision
di: Yuan, Yige, et al.
Pubblicazione: (2025)

Enhancing Temporal Sensitivity of Large Language Model for Recommendation with Counterfactual Tuning
di: Liu, Yutian, et al.
Pubblicazione: (2025)

Towards Understanding the Influence of Reward Margin on Preference Model Performance
di: Qin, Bowen, et al.
Pubblicazione: (2024)

PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization
di: Wu, Jiayi, et al.
Pubblicazione: (2024)

Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
di: Lu, Jinda, et al.
Pubblicazione: (2026)

Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity
di: Fang, Feiteng, et al.
Pubblicazione: (2025)

Preference Poisoning Attacks on Reward Model Learning
di: Wu, Junlin, et al.
Pubblicazione: (2024)

MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
di: Ding, Chenlu, et al.
Pubblicazione: (2025)

RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
di: Jin, Zhuoran, et al.
Pubblicazione: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
di: Wang, Haoxiang, et al.
Pubblicazione: (2024)

Contrastive Weak-to-strong Generalization
di: Jiang, Houcheng, et al.
Pubblicazione: (2025)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
di: Meng, Haoming, et al.
Pubblicazione: (2026)

Word Alignment as Preference for Machine Translation
di: Wu, Qiyu, et al.
Pubblicazione: (2024)

Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets
di: Feng, Duanyu, et al.
Pubblicazione: (2024)

A Common Pitfall of Margin-based Language Model Alignment: Gradient Entanglement
di: Yuan, Hui, et al.
Pubblicazione: (2024)

DiffsFormer: A Diffusion Transformer on Stock Factor Augmentation
di: Gao, Yuan, et al.
Pubblicazione: (2024)

Customizing Language Models with Instance-wise LoRA for Sequential Recommendation
di: Kong, Xiaoyu, et al.
Pubblicazione: (2024)

Beyond the Binary: Capturing Diverse Preferences With Reward Regularization
di: Padmakumar, Vishakh, et al.
Pubblicazione: (2024)

Selective Preference Optimization via Token-Level Reward Function Estimation
di: Yang, Kailai, et al.
Pubblicazione: (2024)

Preference Ranking Optimization for Human Alignment
di: Song, Feifan, et al.
Pubblicazione: (2023)

Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment
di: Yang, Rui, et al.
Pubblicazione: (2024)

Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment
di: Zhang, Hongbin, et al.
Pubblicazione: (2025)

Unified Parameter-Efficient Unlearning for LLMs
di: Ding, Chenlu, et al.
Pubblicazione: (2024)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
di: Tao, Chaofan, et al.
Pubblicazione: (2024)

RLHF in an SFT Way: From Optimal Solution to Reward-Weighted Alignment
di: Du, Yuhao, et al.
Pubblicazione: (2025)