Copertina

T-REG: Preference Optimization with Token-Level Reward Regularization

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhou, Wenxuan, Zhang, Shujian, Zhao, Lingxiao, Meng, Tao
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
di: Zhu, Mingkang, et al.
Pubblicazione: (2025)

Selective Preference Optimization via Token-Level Reward Function Estimation
di: Yang, Kailai, et al.
Pubblicazione: (2024)

WPO: Enhancing RLHF with Weighted Preference Optimization
di: Zhou, Wenxuan, et al.
Pubblicazione: (2024)

SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipe
di: Xiao, Yuxin, et al.
Pubblicazione: (2024)

Optimal Transport-Based Token Weighting scheme for Enhanced Preference Optimization
di: Li, Meng, et al.
Pubblicazione: (2025)

PerPO: Perceptual Preference Optimization via Discriminative Rewarding
di: Zhu, Zining, et al.
Pubblicazione: (2025)

REG: A Regularization Optimizer for Robust Training Dynamics
di: Liu, Zehua, et al.
Pubblicazione: (2025)

Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models
di: Zhang, Wenxuan, et al.
Pubblicazione: (2024)

Iterative Length-Regularized Direct Preference Optimization: A Case Study on Improving 7B Language Models to GPT-4 Level
di: Liu, Jie, et al.
Pubblicazione: (2024)

Beyond Next Token Prediction: Patch-Level Training for Large Language Models
di: Shao, Chenze, et al.
Pubblicazione: (2024)

AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization
di: Wu, Junkang, et al.
Pubblicazione: (2024)

Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards
di: Wang, Haoxiang, et al.
Pubblicazione: (2024)

Token-Level LLM Collaboration via FusionRoute
di: Xiong, Nuoya, et al.
Pubblicazione: (2026)

Preference Poisoning Attacks on Reward Model Learning
di: Wu, Junlin, et al.
Pubblicazione: (2024)

Taming Extreme Tokens: Covariance-Aware GRPO with Gaussian-Kernel Advantage Reweighting
di: Wang, Cheng, et al.
Pubblicazione: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
di: Rafailov, Rafael, et al.
Pubblicazione: (2023)

Multi-Preference Optimization: Generalizing DPO via Set-Level Contrasts
di: Gupta, Taneesh, et al.
Pubblicazione: (2024)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
di: Indurthi, Sathish Reddy, et al.
Pubblicazione: (2024)

ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimization
di: Yoon, Hee Suk, et al.
Pubblicazione: (2025)

Fantastic Reasoning Behaviors and Where to Find Them: Unsupervised Discovery of the Reasoning Process
di: Zhang, Zhenyu, et al.
Pubblicazione: (2025)

Towards Improved Preference Optimization Pipeline: from Data Generation to Budget-Controlled Regularization
di: Chen, Zhuotong, et al.
Pubblicazione: (2024)

Process Rewards with Learned Reliability
di: Li, Jinyuan, et al.
Pubblicazione: (2026)

RewardAnything: Generalizable Principle-Following Reward Models
di: Yu, Zhuohao, et al.
Pubblicazione: (2025)

Beyond Scalar Reward Model: Learning Generative Judge from Preference Data
di: Ye, Ziyi, et al.
Pubblicazione: (2024)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
di: Huang, Kexin, et al.
Pubblicazione: (2025)

Gradient Regularization Prevents Reward Hacking in Reinforcement Learning from Human Feedback and Verifiable Rewards
di: Ackermann, Johannes, et al.
Pubblicazione: (2026)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
di: Meng, Haoming, et al.
Pubblicazione: (2026)

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization
di: Huang, Audrey, et al.
Pubblicazione: (2024)

Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy
di: Liu, Chris Yuhao, et al.
Pubblicazione: (2025)

West-of-N: Synthetic Preferences for Self-Improving Reward Models
di: Pace, Alizée, et al.
Pubblicazione: (2024)

mDPO: Conditional Preference Optimization for Multimodal Large Language Models
di: Wang, Fei, et al.
Pubblicazione: (2024)

TLPO: Token-Level Policy Optimization for Mitigating Language Confusion in Large Language Models
di: Choo, Jinho, et al.
Pubblicazione: (2026)

On the Role of Preference Variance in Preference Optimization
di: Guo, Jiacheng, et al.
Pubblicazione: (2025)

PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models
di: Song, Mingyang, et al.
Pubblicazione: (2025)

AMPO: Active Multi-Preference Optimization for Self-play Preference Selection
di: Gupta, Taneesh, et al.
Pubblicazione: (2025)

LaSeR: Reinforcement Learning with Last-Token Self-Rewarding
di: Yang, Wenkai, et al.
Pubblicazione: (2025)

Direct Reasoning Optimization: Token-Level Reasoning Reflectivity Meets Rubric Gates for Unverifiable Tasks
di: Xu, Yifei, et al.
Pubblicazione: (2025)

Switchable Decision: Dynamic Neural Generation Networks
di: Zhang, Shujian, et al.
Pubblicazione: (2024)

Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
di: Qi, Xuan, et al.
Pubblicazione: (2025)

The Role of Diversity in In-Context Learning for Large Language Models
di: Xiao, Wenyang, et al.
Pubblicazione: (2025)