Copertina

Disentangling Length Bias In Preference Learning Via Response-Conditioned Modeling

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Cai, Jianfeng, Zhu, Jinhua, Sun, Ruopei, Wang, Yue, Li, Li, Zhou, Wengang, Li, Houqiang
Natura:	Preprint
Pubblicazione:	2025
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF
di: Zhao, Kangwen, et al.
Pubblicazione: (2025)

CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
di: Cai, Jianfeng, et al.
Pubblicazione: (2026)

Multi-Level Aware Preference Learning: Enhancing RLHF for Complex Multi-Instruction Tasks
di: Sun, Ruopei, et al.
Pubblicazione: (2025)

Disentangling Length from Quality in Direct Preference Optimization
di: Park, Ryan, et al.
Pubblicazione: (2024)

Explaining Length Bias in LLM-Based Preference Evaluations
di: Hu, Zhengyu, et al.
Pubblicazione: (2024)

Sinkhorn Distance Minimization for Knowledge Distillation
di: Cui, Xiao, et al.
Pubblicazione: (2024)

Search-Based Credit Assignment for Offline Preference-Based Reinforcement Learning
di: Gao, Xiancheng, et al.
Pubblicazione: (2025)

Length Desensitization in Direct Preference Optimization
di: Liu, Wei, et al.
Pubblicazione: (2024)

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
di: Zhang, Zongmeng, et al.
Pubblicazione: (2024)

More Thinking, More Bias: Length-Driven Position Bias in Reasoning Models
di: Wang, Xiao
Pubblicazione: (2026)

BoolQuestions: Does Dense Retrieval Understand Boolean Logic in Language?
di: Zhang, Zongmeng, et al.
Pubblicazione: (2024)

Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs
di: Qin, Yijian, et al.
Pubblicazione: (2023)

Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results
di: Santilli, Andrea, et al.
Pubblicazione: (2025)

On the Role of Preference Variance in Preference Optimization
di: Guo, Jiacheng, et al.
Pubblicazione: (2025)

Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data
di: Ou, Jingyang, et al.
Pubblicazione: (2024)

TRA: Better Length Generalisation with Threshold Relative Attention
di: Opper, Mattia, et al.
Pubblicazione: (2025)

Bias in Large Language Models: Origin, Evaluation, and Mitigation
di: Guo, Yufei, et al.
Pubblicazione: (2024)

Exploiting GPT-4 Vision for Zero-shot Point Cloud Understanding
di: Sun, Qi, et al.
Pubblicazione: (2024)

Knowledge Fusion of Large Language Models Via Modular SkillPacks
di: Du, Guodong, et al.
Pubblicazione: (2025)

Improving Variable-Length Generation in Diffusion Language Models via Length Regularization
di: Cheng, Zicong, et al.
Pubblicazione: (2026)

Semi-Supervised Spoken Language Glossification
di: Yao, Huijie, et al.
Pubblicazione: (2024)

Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
di: Xiao, Teng, et al.
Pubblicazione: (2024)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
di: Ye, Ziyi, et al.
Pubblicazione: (2024)

Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer
di: Dong, Yihe, et al.
Pubblicazione: (2025)

Race, Ethnicity and Their Implication on Bias in Large Language Models
di: Hu, Shiyue, et al.
Pubblicazione: (2026)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
di: Yang, Jinming, et al.
Pubblicazione: (2026)

Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities
di: Hua, Wenyue, et al.
Pubblicazione: (2024)

Robust Multimodal Large Language Models Against Modality Conflict
di: Zhang, Zongmeng, et al.
Pubblicazione: (2025)

Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
di: Peng, Andi, et al.
Pubblicazione: (2024)

P-RAG: Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task
di: Xu, Weiye, et al.
Pubblicazione: (2024)

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
di: Li, Zichao, et al.
Pubblicazione: (2026)

Bayesian Preference Learning for Test-Time Steerable Reward Models
di: Hong, Jiwoo, et al.
Pubblicazione: (2026)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
di: Zhu, Mingkang, et al.
Pubblicazione: (2025)

Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models
di: Cui, Xiao, et al.
Pubblicazione: (2024)

SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
di: Hu, Chenzhi, et al.
Pubblicazione: (2026)

Eliminating Position Bias of Language Models: A Mechanistic Approach
di: Wang, Ziqi, et al.
Pubblicazione: (2024)

Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
di: Leng, Jiaqi, et al.
Pubblicazione: (2025)

MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
di: Xia, Xinfeng, et al.
Pubblicazione: (2025)

Preference Leakage: A Contamination Problem in LLM-as-a-judge
di: Li, Dawei, et al.
Pubblicazione: (2025)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
di: Wang, Xiaobo, et al.
Pubblicazione: (2025)