Cover Image

Token-weighted Direct Preference Optimization with Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Huang, Chengyu, Li, Zhuohang, Chou, Sheng-Yen, Cardie, Claire
Format:	Preprint
Published:	2026
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Bootstrapping Post-training Signals for Open-ended Tasks via Rubric-based Self-play on Pre-training Text
by: Huang, Chengyu, et al.
Published: (2026)

HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimization
by: Huang, Chengyu, et al.
Published: (2025)

Better LLM Reasoning via Dual-Play
by: Zhang, Zhengxin, et al.
Published: (2025)

CLIPer: Tailoring Diverse User Preference via Classifier-Guided Inference-Time Personalization
by: Su, Jinyan, et al.
Published: (2026)

Knowing but Not Showing: LLMs Recognize Ambiguity but Rarely Ask Clarifying Questions
by: Su, Jinyan, et al.
Published: (2026)

Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards
by: Su, Jinyan, et al.
Published: (2025)

Reasoning Court: Combining Reasoning, Action, and Judgment for Multi-Hop Reasoning
by: Wu, Jingtian, et al.
Published: (2025)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

MASH: Modeling Abstention via Selective Help-Seeking
by: Gul, Mustafa Omer, et al.
Published: (2025)

Adapting Fake News Detection to the Era of Large Language Models
by: Su, Jinyan, et al.
Published: (2023)

DCRM: A Heuristic to Measure Response Pair Quality in Preference Optimization
by: Huang, Chengyu, et al.
Published: (2025)

Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective
by: Shao, Ruichen, et al.
Published: (2025)

Judging with Confidence: Calibrating Autoraters to Preference Distributions
by: Li, Zhuohang, et al.
Published: (2025)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
by: Su, Jinyan, et al.
Published: (2025)

Direct Judgement Preference Optimization
by: Wang, Peifeng, et al.
Published: (2024)

I Could've Asked That: Reformulating Unanswerable Questions
by: Zhao, Wenting, et al.
Published: (2024)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
by: Zhu, Mingkang, et al.
Published: (2025)

Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization
by: Jiang, Yuxin, et al.
Published: (2024)

New Desiderata for Direct Preference Optimization
by: Hu, Xiangkun, et al.
Published: (2024)

DPO-Shift: Shifting the Distribution of Direct Preference Optimization
by: Yang, Xiliang, et al.
Published: (2025)

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
by: Li, Jian, et al.
Published: (2025)

DGPO: Beyond Pairwise Preferences with Directional Consistent Groupwise Optimization
by: Deng, Mengyi, et al.
Published: (2026)

On Extending Direct Preference Optimization to Accommodate Ties
by: Chen, Jinghong, et al.
Published: (2024)

SDPO: Segment-Level Direct Preference Optimization for Social Agents
by: Kong, Aobo, et al.
Published: (2025)

WildChat: 1M ChatGPT Interaction Logs in the Wild
by: Zhao, Wenting, et al.
Published: (2024)

Are Triggers Needed for Document-Level Event Extraction?
by: Shaar, Shaden, et al.
Published: (2024)

TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
by: Liu, Aiwei, et al.
Published: (2024)

TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
by: Nguyen, Truong, et al.
Published: (2026)

Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning
by: Wang, Tianduo, et al.
Published: (2024)

TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation
by: Fodeh, Samah, et al.
Published: (2026)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

Filtered Direct Preference Optimization
by: Morimura, Tetsuro, et al.
Published: (2024)

Direct Preference Optimization with an Offset
by: Amini, Afra, et al.
Published: (2024)

Selective Preference Optimization via Token-Level Reward Function Estimation
by: Yang, Kailai, et al.
Published: (2024)

Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
by: Ghassabi, Mehrdad, et al.
Published: (2026)

PHOENIX: Open-Source Language Adaption for Direct Preference Optimization
by: Uhlig, Matthias, et al.
Published: (2024)

MCM-DPO: Multifaceted Cross-Modal Direct Preference Optimization for Alt-text Generation
by: Fu, Jinlan, et al.
Published: (2025)

Understanding Reference Policies in Direct Preference Optimization
by: Liu, Yixin, et al.
Published: (2024)

Accelerating Direct Preference Optimization with Prefix Sharing
by: Wang, Franklin, et al.
Published: (2024)