Image de couverture de livre

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhu, Wenhong, He, Zhiwei, Wang, Xiaofeng, Liu, Pengfei, Wang, Rui
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Weak-to-Strong Reasoning
par: Yang, Yuqing, et autres
Publié: (2024)

Do Large Language Models Truly Understand Geometric Structures?
par: Wang, Xiaofeng, et autres
Publié: (2025)

When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
par: Le, Khoi, et autres
Publié: (2026)

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization
par: Lyu, Yougang, et autres
Publié: (2024)

VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
par: Lu, Xingyu, et autres
Publié: (2026)

Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation
par: Zou, Deyu, et autres
Publié: (2025)

Weak-to-Strong Jailbreaking on Large Language Models
par: Zhao, Xuandong, et autres
Publié: (2024)

MrRoPE: Mixed-radix Rotary Position Embedding
par: Tian, Qingyuan, et autres
Publié: (2026)

Improving Open-Ended Text Generation via Adaptive Decoding
par: Zhu, Wenhong, et autres
Publié: (2024)

Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
par: Song, Feifan, et autres
Publié: (2025)

Hybrid Policy Distillation for LLMs
par: Zhu, Wenhong, et autres
Publié: (2026)

Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
par: Yang, Wenkai, et autres
Publié: (2024)

Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
par: Zakershahrak, Mehrdad, et autres
Publié: (2024)

Reward Modeling with Weak Supervision for Language Models
par: Hauptvogel, Ben, et autres
Publié: (2024)

GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulation
par: He, Jie, et autres
Publié: (2025)

Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
par: Li, Bolian, et autres
Publié: (2025)

Adding Alignment Control to Language Models
par: Zhu, Wenhong, et autres
Publié: (2025)

W2S-AlignTree: Weak-to-Strong Inference-Time Alignment for Large Language Models via Monte Carlo Tree Search
par: Ding, Zhenyu, et autres
Publié: (2025)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
par: Zhou, Zhanhui, et autres
Publié: (2024)

Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
par: Li, Ming, et autres
Publié: (2024)

Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning
par: Sang, Jitao, et autres
Publié: (2024)

Incentivizing Strong Reasoning from Weak Supervision
par: Yuan, Yige, et autres
Publié: (2025)

The Alignment Floor: How Persona Customization Breaks Safety in Weakly-Aligned LLMs
par: Zhang, Xing, et autres
Publié: (2026)

Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
par: da Silva, Italo Luis, et autres
Publié: (2024)

Optimizing Language Model's Reasoning Abilities with Weak Supervision
par: Tong, Yongqi, et autres
Publié: (2024)

Selective Weak-to-Strong Generalization
par: Lang, Hao, et autres
Publié: (2025)

Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
par: Ai, Yiming, et autres
Publié: (2024)

User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction
par: Hao, Yuren, et autres
Publié: (2026)

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation
par: Cui, Guofeng, et autres
Publié: (2025)

Theoretical Analysis of Weak-to-Strong Generalization
par: Lang, Hunter, et autres
Publié: (2024)

Debate Helps Weak-to-Strong Generalization
par: Lang, Hao, et autres
Publié: (2025)

How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?
par: Cui, Yingqian, et autres
Publié: (2026)

Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
par: Jin, Zhuoran, et autres
Publié: (2025)

AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference
par: Han, Yang, et autres
Publié: (2024)

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
par: Liu, Kaiyuan, et autres
Publié: (2026)

Proximal Supervised Fine-Tuning
par: Zhu, Wenhong, et autres
Publié: (2025)

Flexible Realignment of Language Models
par: Zhu, Wenhong, et autres
Publié: (2025)

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack
par: Fu, Yu, et autres
Publié: (2023)

CLEAN-EVAL: Clean Evaluation on Contaminated Large Language Models
par: Zhu, Wenhong, et autres
Publié: (2023)

Synthesizing Text-to-SQL Data from Weak and Strong LLMs
par: Yang, Jiaxi, et autres
Publié: (2024)