Cover Image

Orthogonal Finetuning for Direct Preference Optimization

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Chenxu, Jia, Ruipeng, Gu, Naibin, Lin, Zheng, Chen, Siyuan, Pang, Chao, Yin, Weichong, Sun, Yu, Wu, Hua, Wang, Weiping
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Weights-Rotated Preference Optimization for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)

Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
by: Liu, Xiyu, et al.
Published: (2025)

Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models
by: Liu, Xiyu, et al.
Published: (2026)

Advantageous Parameter Expansion Training Makes Better Large Language Models
by: Gu, Naibin, et al.
Published: (2025)

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning
by: Gu, Naibin, et al.
Published: (2024)

Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models
by: Gu, Naibin, et al.
Published: (2025)

Self-Distilled RLVR
by: Yang, Chenxu, et al.
Published: (2026)

BeamLoRA: Beam-Constraint Low-Rank Adaptation
by: Gu, Naibin, et al.
Published: (2025)

DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts
by: Feng, Yuchen, et al.
Published: (2025)

CBP-Tuning: Efficient Local Customization for Black-box Large Language Models
by: Zhao, Jiaxuan, et al.
Published: (2025)

Relation Also Knows: Rethinking the Recall and Editing of Factual Associations in Auto-Regressive Transformer Language Models
by: Liu, Xiyu, et al.
Published: (2024)

Elastic MoE: Unlocking the Inference-Time Scalability of Mixture-of-Experts
by: Gu, Naibin, et al.
Published: (2025)

A Multi-Task Role-Playing Agent Capable of Imitating Character Linguistic Styles
by: Chen, Siyuan, et al.
Published: (2024)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

ATTNPO: Attention-Guided Process Supervision for Efficient Reasoning
by: Nie, Shuaiyi, et al.
Published: (2026)

On Extending Direct Preference Optimization to Accommodate Ties
by: Chen, Jinghong, et al.
Published: (2024)

AdaTIR: Adaptive Tool-Integrated Reasoning via Difficulty-Aware Policy Optimization
by: Fang, Zhaiyu, et al.
Published: (2026)

Enhancing LLM Safety via Constrained Direct Preference Optimization
by: Liu, Zixuan, et al.
Published: (2024)

Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding
by: Yang, Guangyu, et al.
Published: (2023)

Orthogonal Finetuning Made Scalable
by: Qiu, Zeju, et al.
Published: (2025)

Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?
by: Wang, Yatong, et al.
Published: (2026)

Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
by: Li, Jian, et al.
Published: (2025)

A Factuality and Diversity Reconciled Decoding Method for Knowledge-Grounded Dialogue Generation
by: Yang, Chenxu, et al.
Published: (2024)

Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs
by: Zhang, Xuan, et al.
Published: (2024)

KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance
by: Yu, Linhao, et al.
Published: (2026)

VecInfer: Efficient LLM Inference with Low-Bit KV Cache via Outlier-Suppressed Vector Quantization
by: Yao, Dingyu, et al.
Published: (2025)

Test-time Prompt Intervention
by: Yang, Chenxu, et al.
Published: (2025)

Direct Judgement Preference Optimization
by: Wang, Peifeng, et al.
Published: (2024)

TGDPO: Harnessing Token-Level Reward Guidance for Enhancing Direct Preference Optimization
by: Zhu, Mingkang, et al.
Published: (2025)

Token-level Direct Preference Optimization
by: Zeng, Yongcheng, et al.
Published: (2024)

Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence
by: Lu, Junru, et al.
Published: (2024)

Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubric
by: Jia, Ruipeng, et al.
Published: (2026)

Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
by: Liu, Weiyang, et al.
Published: (2023)

Length Desensitization in Direct Preference Optimization
by: Liu, Wei, et al.
Published: (2024)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

DPO-Shift: Shifting the Distribution of Direct Preference Optimization
by: Yang, Xiliang, et al.
Published: (2025)

OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning
by: Feng, Jinyuan, et al.
Published: (2025)

New Desiderata for Direct Preference Optimization
by: Hu, Xiangkun, et al.
Published: (2024)

2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision
by: Li, Shilong, et al.
Published: (2024)

Breaking the Trade-Off Between Faithfulness and Expressiveness for Large Language Models
by: Yang, Chenxu, et al.
Published: (2025)