Cover Image

PORT: Preference Optimization on Reasoning Traces

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lahlou, Salem, Abubaker, Abdalgader, Hacid, Hakim
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNets
by: Younsi, Adam, et al.
Published: (2025)

Investigating Regularization of Self-Play Language Models
by: Alami, Reda, et al.
Published: (2024)

WavLink: Compact Audio-Text Embeddings with a Global Whisper Token
by: Kumar, Gokul Karthik, et al.
Published: (2026)

ArabicDialectHub: A Cross-Dialectal Arabic Learning Resource and Platform
by: Lahlou, Salem
Published: (2026)

From Noisy Traces to Stable Gradients: Bias-Variance Optimized Preference Optimization for Aligning Large Reasoning Models
by: Zhu, Mingkang, et al.
Published: (2025)

Re-thinking Human Activity Recognition with Hierarchy-aware Label Relationship Modeling
by: Zuo, Jingwei, et al.
Published: (2024)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
by: Zhang, Xuan, et al.
Published: (2024)

Competitive Audio-Language Models with Data-Efficient Single-Stage Training on Public Data
by: Kumar, Gokul Karthik, et al.
Published: (2025)

Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimization
by: Kawakami, Wataru, et al.
Published: (2025)

SD-E$^2$: Semantic Exploration for Reasoning Under Token Budgets
by: Mishra, Kshitij, et al.
Published: (2026)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

Preference Optimization with Multi-Sample Comparisons
by: Wang, Chaoqi, et al.
Published: (2024)

SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
by: Elbadry, Rania, et al.
Published: (2026)

Communication Enables Cooperation in LLM Agents: A Comparison with Curriculum-Based Approaches
by: Madmoun, Hachem, et al.
Published: (2025)

Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs
by: Lai, Xin, et al.
Published: (2024)

Self-Evolved Preference Optimization for Enhancing Mathematical Reasoning in Small Language Models
by: Singh, Joykirat, et al.
Published: (2025)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

Understanding Reference Policies in Direct Preference Optimization
by: Liu, Yixin, et al.
Published: (2024)

Accelerating Direct Preference Optimization with Prefix Sharing
by: Wang, Franklin, et al.
Published: (2024)

Scaling Policy Compliance Assessment in Language Models with Policy Reasoning Traces
by: Imperial, Joseph Marvin, et al.
Published: (2025)

Beyond Preferences: Learning Alignment Principles Grounded in Human Reasons and Values
by: Bell, Henry, et al.
Published: (2026)

Alignment with Preference Optimization Is All You Need for LLM Safety
by: Alami, Reda, et al.
Published: (2024)

Group Robust Preference Optimization in Reward-free RLHF
by: Ramesh, Shyam Sundhar, et al.
Published: (2024)

Direct Multi-Turn Preference Optimization for Language Agents
by: Shi, Wentao, et al.
Published: (2024)

Multi-Reference Preference Optimization for Large Language Models
by: Le, Hung, et al.
Published: (2024)

The Crucial Role of Samplers in Online Direct Preference Optimization
by: Shi, Ruizhe, et al.
Published: (2024)

Robust Preference Optimization through Reward Model Distillation
by: Fisch, Adam, et al.
Published: (2024)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

Multi-Response Preference Optimization with Augmented Ranking Dataset
by: Gwon, Hansle, et al.
Published: (2024)

Intrinsic Mutual Information as a Modulator for Preference Optimization
by: Liao, Peng, et al.
Published: (2026)

Adaptive Preference Optimization with Uncertainty-aware Utility Anchor
by: Wang, Xiaobo, et al.
Published: (2025)

Tracing Uncertainty in Language Model "Reasoning"
by: Grünefeld, Nils, et al.
Published: (2026)

Geometric-Averaged Preference Optimization for Soft Preference Labels
by: Furuta, Hiroki, et al.
Published: (2024)

Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
by: Liu, Shuo, et al.
Published: (2026)

$f$-PO: Generalizing Preference Optimization with $f$-divergence Minimization
by: Han, Jiaqi, et al.
Published: (2024)

LiPO: Listwise Preference Optimization through Learning-to-Rank
by: Liu, Tianqi, et al.
Published: (2024)

Enhancing LLM Safety via Constrained Direct Preference Optimization
by: Liu, Zixuan, et al.
Published: (2024)

Offline Preference Optimization via Maximum Marginal Likelihood Estimation
by: Najafi, Saeed, et al.
Published: (2025)

Semantic Communication meets System 2 ML: How Abstraction, Compositionality and Emergent Languages Shape Intelligence
by: Bennis, Mehdi, et al.
Published: (2025)

Filtered Direct Preference Optimization
by: Morimura, Tetsuro, et al.
Published: (2024)