Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling
Fuente:
arXiv
Saved in:
| Main Authors: | Cai, Jianfeng, Zhu, Jinhua, Sun, Ruopei, Wang, Yue, Li, Li, Zhou, Wengang, Li, Houqiang |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Similar Items
Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
by: Zhao, Kangwen, et al.
Published: (2025)
by: Zhao, Kangwen, et al.
Published: (2025)
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
by: Cai, Jianfeng, et al.
Published: (2026)
by: Cai, Jianfeng, et al.
Published: (2026)
Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks
by: Sun, Ruopei, et al.
Published: (2025)
by: Sun, Ruopei, et al.
Published: (2025)
Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)
by: Park, Ryan, et al.
Published: (2024)
Explaining Length Bias in LLM-Based Preference Evaluations
by: Hu, Zhengyu, et al.
Published: (2024)
by: Hu, Zhengyu, et al.
Published: (2024)
Sinkhorn Distance Minimization for Knowledge Distillation
by: Cui, Xiao, et al.
Published: (2024)
by: Cui, Xiao, et al.
Published: (2024)
Search-Based Credit Assignment for Offline Preference-Based Reinforcement Learning
by: Gao, Xiancheng, et al.
Published: (2025)
by: Gao, Xiancheng, et al.
Published: (2025)
Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)
by: Liu, Wei, et al.
Published: (2024)
Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
by: Zhang, Zongmeng, et al.
Published: (2024)
by: Zhang, Zongmeng, et al.
Published: (2024)
More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models
by: Wang, Xiao
Published: (2026)
by: Wang, Xiao
Published: (2026)
BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language?
by: Zhang, Zongmeng, et al.
Published: (2024)
by: Zhang, Zongmeng, et al.
Published: (2024)
Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs
by: Qin, Yijian, et al.
Published: (2023)
by: Qin, Yijian, et al.
Published: (2023)
Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results
by: Santilli, Andrea, et al.
Published: (2025)
by: Santilli, Andrea, et al.
Published: (2025)
On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)
by: Guo, Jiacheng, et al.
Published: (2025)
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
by: Ou, Jingyang, et al.
Published: (2024)
by: Ou, Jingyang, et al.
Published: (2024)
TRA: Better Length Generalisation with Threshold Relative Attention
by: Opper, Mattia, et al.
Published: (2025)
by: Opper, Mattia, et al.
Published: (2025)
Bias in Large Language Models: Origin, Evaluation, and Mitigation
by: Guo, Yufei, et al.
Published: (2024)
by: Guo, Yufei, et al.
Published: (2024)
Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
by: Sun, Qi, et al.
Published: (2024)
by: Sun, Qi, et al.
Published: (2024)
Knowledge Fusion of Large Language Models Via Modular SkillPacks
by: Du, Guodong, et al.
Published: (2025)
by: Du, Guodong, et al.
Published: (2025)
Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
by: Cheng, Zicong, et al.
Published: (2026)
by: Cheng, Zicong, et al.
Published: (2026)
Semi-Supervised Spoken Language Glossification
by: Yao, Huijie, et al.
Published: (2024)
by: Yao, Huijie, et al.
Published: (2024)
Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
by: Xiao, Teng, et al.
Published: (2024)
by: Xiao, Teng, et al.
Published: (2024)
Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
by: Ye, Ziyi, et al.
Published: (2024)
by: Ye, Ziyi, et al.
Published: (2024)
Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
by: Dong, Yihe, et al.
Published: (2025)
by: Dong, Yihe, et al.
Published: (2025)
Race, Ethnicity and Their Implication on Bias in Large Language Models
by: Hu, Shiyue, et al.
Published: (2026)
by: Hu, Shiyue, et al.
Published: (2026)
Quantifying and Mitigating Self-Preference Bias of LLM Judges
by: Yang, Jinming, et al.
Published: (2026)
by: Yang, Jinming, et al.
Published: (2026)
Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities
by: Hua, Wenyue, et al.
Published: (2024)
by: Hua, Wenyue, et al.
Published: (2024)
Robust Multimodal Large Language Models Against Modality Conflict
by: Zhang, Zongmeng, et al.
Published: (2025)
by: Zhang, Zongmeng, et al.
Published: (2025)
Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
by: Peng, Andi, et al.
Published: (2024)
by: Peng, Andi, et al.
Published: (2024)
P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task
by: Xu, Weiye, et al.
Published: (2024)
by: Xu, Weiye, et al.
Published: (2024)
Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
by: Li, Zichao, et al.
Published: (2026)
by: Li, Zichao, et al.
Published: (2026)
Bayesian Preference Learning for Test-Time Steerable Reward Models
by: Hong, Jiwoo, et al.
Published: (2026)
by: Hong, Jiwoo, et al.
Published: (2026)
From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
by: Zhu, Mingkang, et al.
Published: (2025)
by: Zhu, Mingkang, et al.
Published: (2025)
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
by: Cui, Xiao, et al.
Published: (2024)
by: Cui, Xiao, et al.
Published: (2024)
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
by: Hu, Chenzhi, et al.
Published: (2026)
by: Hu, Chenzhi, et al.
Published: (2026)
Eliminating Position Bias of Language Models: A Mechanistic Approach
by: Wang, Ziqi, et al.
Published: (2024)
by: Wang, Ziqi, et al.
Published: (2024)
Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
by: Leng, Jiaqi, et al.
Published: (2025)
by: Leng, Jiaqi, et al.
Published: (2025)
MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
by: Xia, Xinfeng, et al.
Published: (2025)
by: Xia, Xinfeng, et al.
Published: (2025)
Preference Leakage: A Contamination Problem in LLM-as-a-judge
by: Li, Dawei, et al.
Published: (2025)
by: Li, Dawei, et al.
Published: (2025)
Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
by: Wang, Xiaobo, et al.
Published: (2025)
by: Wang, Xiaobo, et al.
Published: (2025)
Similar Items
-
Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
by: Zhao, Kangwen, et al.
Published: (2025) -
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
by: Cai, Jianfeng, et al.
Published: (2026) -
Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks
by: Sun, Ruopei, et al.
Published: (2025) -
Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024) -
Explaining Length Bias in LLM-Based Preference Evaluations
by: Hu, Zhengyu, et al.
Published: (2024)