Cover Image

Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Song, Feifan, Wei, Shaohang, Luo, Wen, Fan, Yuxuan, Liu, Tianyu, Wang, Guoyin, Wang, Houfeng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Odysseus Navigates the Sirens' Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation
by: Luo, Wen, et al.
Published: (2025)

Well Begun, Half Done: Reinforcement Learning with Prefix Optimization for LLM Reasoning
by: Sun, Yiliu, et al.
Published: (2025)

ICDPO: Effectively Borrowing Alignment Capability of Others via In-context Direct Preference Optimization
by: Song, Feifan, et al.
Published: (2024)

P-Aligner: Enabling Pre-Alignment of Language Models via Principled Instruction Synthesis
by: Song, Feifan, et al.
Published: (2025)

Noise-Level Diffusion Guidance: Well Begun is Half Done
by: Mannering, Harvey, et al.
Published: (2025)

TIME: A Multi-level Benchmark for Temporal Reasoning of LLMs in Real-World Scenarios
by: Wei, Shaohang, et al.
Published: (2025)

Preference Ranking Optimization for Human Alignment
by: Song, Feifan, et al.
Published: (2023)

Only Say What You Know: Calibration-Aware Generation for Long-Form Factuality
by: Luo, Wen, et al.
Published: (2026)

Mitigating Overthinking through Reasoning Shaping
by: Song, Feifan, et al.
Published: (2025)

Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations
by: Luo, Wen, et al.
Published: (2026)

Well Begun is Half Done: Location-Aware and Trace-Guided Iterative Automated Vulnerability Repair
by: Ye, Zhenlei, et al.
Published: (2025)

Well Begun is Half Done: Training-Free and Model-Agnostic Semantically Guaranteed User Representation Initialization for Multimodal Recommendation
by: Xu, Jinfeng, et al.
Published: (2026)

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment
by: Song, Feifan, et al.
Published: (2024)

Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction
by: Li, Wei, et al.
Published: (2025)

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model
by: Zhu, Wenhong, et al.
Published: (2024)

Detection-Correction Structure via General Language Model for Grammatical Error Correction
by: Li, Wei, et al.
Published: (2024)

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization
by: Lyu, Yougang, et al.
Published: (2024)

PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation
by: Liu, Yuxuan
Published: (2025)

Investigating the (De)Composition Capabilities of Large Language Models in Natural-to-Formal Language Conversion
by: Xu, Ziyao, et al.
Published: (2025)

SPOR: A Comprehensive and Practical Evaluation Method for Compositional Generalization in Data-to-Text Generation
by: Xu, Ziyao, et al.
Published: (2024)

Batch Speculative Decoding Done Right
by: Zhang, Ranran Haoran, et al.
Published: (2025)

CiteCheck: Towards Accurate Citation Faithfulness Detection
by: Xu, Ziyao, et al.
Published: (2025)

Weak-to-Strong Jailbreaking on Large Language Models
by: Zhao, Xuandong, et al.
Published: (2024)

Utilizing Local Hierarchy with Adversarial Training for Hierarchical Text Classification
by: Wang, Zihan, et al.
Published: (2024)

On-the-fly Preference Alignment via Principle-Guided Decoding
by: Zhu, Mingye, et al.
Published: (2025)

Improving Weak-to-Strong Generalization with Reliability-Aware Alignment
by: Guo, Yue, et al.
Published: (2024)

Towards a Unified View of Preference Learning for Large Language Models: A Survey
by: Gao, Bofei, et al.
Published: (2024)

HalluDial: A Large-Scale Benchmark for Automatic Dialogue-Level Hallucination Evaluation
by: Luo, Wen, et al.
Published: (2024)

Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
by: Yang, Wenkai, et al.
Published: (2024)

Sparse but Critical: A Token-Level Analysis of Distributional Shifts in RLVR Fine-Tuning of LLMs
by: Meng, Haoming, et al.
Published: (2026)

Your Weak LLM is Secretly a Strong Teacher for Alignment
by: Tao, Leitian, et al.
Published: (2024)

Drift: Decoding-time Personalized Alignments with Implicit User Preferences
by: Kim, Minbeom, et al.
Published: (2025)

Decoding the Ear: A Framework for Objectifying Expressiveness from Human Preference Through Efficient Alignment
by: Lin, Zhiyu, et al.
Published: (2025)

PACIFIC: Can LLMs Discern the Traits Influencing Your Preferences? Evaluating Personality-Driven Preference Alignment in LLMs
by: Zhao, Tianyu, et al.
Published: (2026)

ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantation
by: Dong, Weilong, et al.
Published: (2024)

COIG-P: A High-Quality and Large-Scale Chinese Preference Dataset for Alignment with Human Values
by: P Team, et al.
Published: (2025)

VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
by: Lu, Xingyu, et al.
Published: (2026)

StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs
by: Song, Yuhan, et al.
Published: (2025)

RACE-Align: Retrieval-Augmented and Chain-of-Thought Enhanced Preference Alignment for Large Language Models
by: Yan, Qihang, et al.
Published: (2025)

A Systematic Evaluation of Preference Aggregation in Federated RLHF for Pluralistic Alignment of LLMs
by: Srewa, Mahmoud, et al.
Published: (2025)