Cover Image

Selective Preference Optimization via Token-Level Reward Function Estimation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Kailai, Liu, Zhiwei, Xie, Qianqian, Huang, Jimin, Min, Erxue, Ananiadou, Sophia
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models
by: Yang, Kailai, et al.
Published: (2024)

T-REG: Preference Optimization with Token-Level Reward Regularization
by: Zhou, Wenxuan, et al.
Published: (2024)

MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs
by: Liu, Zhiwei, et al.
Published: (2025)

FMDLlama: Financial Misinformation Detection based on Large Language Models
by: Liu, Zhiwei, et al.
Published: (2024)

LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisation
by: Bishop, Jennifer A, et al.
Published: (2023)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
by: Zhu, Mingkang, et al.
Published: (2025)

HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection
by: Wang, Yuxin, et al.
Published: (2024)

MemAdapter: Fast Alignment across Agent Memory Paradigms via Generative Subgraph Retrieval
by: Zhang, Xin, et al.
Published: (2026)

EmoLLMs: A Series of Emotional Large Language Models and Annotation Tools for Comprehensive Affective Analysis
by: Liu, Zhiwei, et al.
Published: (2024)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
by: Zhu, Zining, et al.
Published: (2025)

MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models
by: Yang, Kailai, et al.
Published: (2023)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
by: Huang, Kexin, et al.
Published: (2025)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
by: Yoon, Hee Suk, et al.
Published: (2025)

LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
by: Yang, Wenkai, et al.
Published: (2025)

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
by: Nguyen, Truong, et al.
Published: (2026)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
by: Wu, Junkang, et al.
Published: (2024)

HealMe: Harnessing Cognitive Reframing in Large Language Models for Psychotherapy
by: Xiao, Mengxi, et al.
Published: (2024)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability
by: Lin, Zicheng, et al.
Published: (2024)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
by: Qi, Xuan, et al.
Published: (2025)

Rumor Detection by Multi-task Suffix Learning based on Time-series Dual Sentiments
by: Liu, Zhiwei, et al.
Published: (2025)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
by: Gupta, Taneesh, et al.
Published: (2024)

RAEmoLLM: Retrieval Augmented LLMs for Cross-Domain Misinformation Detection Using In-Context Learning Based on Emotional Information
by: Liu, Zhiwei, et al.
Published: (2024)

Natural Language Processing for Cardiology: A Narrative Review
by: Yang, Kailai, et al.
Published: (2025)

DavIR: Data Selection via Implicit Reward for Large Language Models
by: Zhou, Haotian, et al.
Published: (2023)

RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution
by: Li, Jiahui, et al.
Published: (2024)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
by: Liu, Chris Yuhao, et al.
Published: (2025)

Token-Level LLM Collaboration via FusionRoute
by: Xiong, Nuoya, et al.
Published: (2026)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
by: Gupta, Taneesh, et al.
Published: (2025)

Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
by: Li, Meng, et al.
Published: (2025)

Influential Language Data Selection via Gradient Trajectory Pursuit
by: Deng, Zhiwei, et al.
Published: (2024)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

Preference Optimization by Estimating the Ratio of the Data Distribution
by: Kim, Yeongmin, et al.
Published: (2025)

MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
by: Saha, Anisha, et al.
Published: (2026)

Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training
by: Yang, Kailai, et al.
Published: (2025)

TIPS: Turn-Level Information-Potential Reward Shaping for Search-Augmented LLMs
by: Xie, Yutao, et al.
Published: (2026)

From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling
by: Kabir, Mohsinul, et al.
Published: (2025)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
by: Wang, Haoxiang, et al.
Published: (2024)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
by: Jian, Chengtao, et al.
Published: (2025)