Copertina

Weak-to-Strong Preference Optimization: Stealing Reward from Weak Aligned Model

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhu, Wenhong, He, Zhiwei, Wang, Xiaofeng, Liu, Pengfei, Wang, Rui
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Weak-to-Strong Reasoning
di: Yang, Yuqing, et al.
Pubblicazione: (2024)

Do Large Language Models Truly Understand Geometric Structures?
di: Wang, Xiaofeng, et al.
Pubblicazione: (2025)

When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
di: Le, Khoi, et al.
Pubblicazione: (2026)

MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization
di: Lyu, Yougang, et al.
Pubblicazione: (2024)

VCap: Hypergeometric Rewards for Weak-to-Strong Visual Captioning
di: Lu, Xingyu, et al.
Pubblicazione: (2026)

Weak-to-Strong GraphRAG: Aligning Weak Retrievers with Large Language Models for Graph-based Retrieval Augmented Generation
di: Zou, Deyu, et al.
Pubblicazione: (2025)

Weak-to-Strong Jailbreaking on Large Language Models
di: Zhao, Xuandong, et al.
Pubblicazione: (2024)

Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding
di: Song, Feifan, et al.
Pubblicazione: (2025)

MrRoPE: Mixed-radix Rotary Position Embedding
di: Tian, Qingyuan, et al.
Pubblicazione: (2026)

Improving Open-Ended Text Generation via Adaptive Decoding
di: Zhu, Wenhong, et al.
Pubblicazione: (2024)

Hybrid Policy Distillation for LLMs
di: Zhu, Wenhong, et al.
Pubblicazione: (2026)

Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalization
di: Yang, Wenkai, et al.
Pubblicazione: (2024)

Explanation, Debate, Align: A Weak-to-Strong Framework for Language Model Generalization
di: Zakershahrak, Mehrdad, et al.
Pubblicazione: (2024)

Reward Modeling with Weak Supervision for Language Models
di: Hauptvogel, Ben, et al.
Pubblicazione: (2024)

GenTool: Enhancing Tool Generalization in Language Models through Zero-to-One and Weak-to-Strong Simulation
di: He, Jie, et al.
Pubblicazione: (2025)

Reward-Shifted Speculative Sampling Is An Efficient Test-Time Weak-to-Strong Aligner
di: Li, Bolian, et al.
Pubblicazione: (2025)

Adding Alignment Control to Language Models
di: Zhu, Wenhong, et al.
Pubblicazione: (2025)

W2S-AlignTree: Weak-to-Strong Inference-Time Alignment for Large Language Models via Monte Carlo Tree Search
di: Ding, Zhenyu, et al.
Pubblicazione: (2025)

Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models
di: Zhou, Zhanhui, et al.
Pubblicazione: (2024)

Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
di: Li, Ming, et al.
Pubblicazione: (2024)

Improving Weak-to-Strong Generalization with Scalable Oversight and Ensemble Learning
di: Sang, Jitao, et al.
Pubblicazione: (2024)

Incentivizing Strong Reasoning from Weak Supervision
di: Yuan, Yige, et al.
Pubblicazione: (2025)

The Alignment Floor: How Persona Customization Breaks Safety in Weakly-Aligned LLMs
di: Zhang, Xing, et al.
Pubblicazione: (2026)

Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems
di: da Silva, Italo Luis, et al.
Pubblicazione: (2024)

Optimizing Language Model's Reasoning Abilities with Weak Supervision
di: Tong, Yongqi, et al.
Pubblicazione: (2024)

Selective Weak-to-Strong Generalization
di: Lang, Hao, et al.
Pubblicazione: (2025)

Is Self-knowledge and Action Consistent or Not: Investigating Large Language Model's Personality
di: Ai, Yiming, et al.
Pubblicazione: (2024)

User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction
di: Hao, Yuren, et al.
Pubblicazione: (2026)

CRPO: Confidence-Reward Driven Preference Optimization for Machine Translation
di: Cui, Guofeng, et al.
Pubblicazione: (2025)

Theoretical Analysis of Weak-to-Strong Generalization
di: Lang, Hunter, et al.
Pubblicazione: (2024)

Debate Helps Weak-to-Strong Generalization
di: Lang, Hao, et al.
Pubblicazione: (2025)

How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?
di: Cui, Yingqian, et al.
Pubblicazione: (2026)

Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillation
di: Liu, Kaiyuan, et al.
Pubblicazione: (2026)

Omni-Reward: Towards Generalist Omni-Modal Reward Modeling with Free-Form Preferences
di: Jin, Zhuoran, et al.
Pubblicazione: (2025)

AlignSum: Data Pyramid Hierarchical Fine-tuning for Aligning with Human Summarization Preference
di: Han, Yang, et al.
Pubblicazione: (2024)

Flexible Realignment of Language Models
di: Zhu, Wenhong, et al.
Pubblicazione: (2025)

Proximal Supervised Fine-Tuning
di: Zhu, Wenhong, et al.
Pubblicazione: (2025)

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack
di: Fu, Yu, et al.
Pubblicazione: (2023)

CLEAN-EVAL: Clean Evaluation on Contaminated Large Language Models
di: Zhu, Wenhong, et al.
Pubblicazione: (2023)

Synthesizing Text-to-SQL Data from Weak and Strong LLMs
di: Yang, Jiaxi, et al.
Pubblicazione: (2024)