Cover Image

Revisiting Self-Play Preference Optimization: On the Role of Prompt Difficulty

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xiao, Yao, Kim, Jung-jae, Lee, Roy Ka-wei, Bing, Lidong
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Finding the Sweet Spot: Preference Data Construction for Scaling Preference Optimization
by: Xiao, Yao, et al.
Published: (2025)

Document Reconstruction Unlocks Scalable Long-Context RLVR
by: Xiao, Yao, et al.
Published: (2026)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
by: Chen, Guanzheng, et al.
Published: (2025)

Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
by: Wang, Jianyu, et al.
Published: (2025)

ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations
by: Xiao, Yunze, et al.
Published: (2024)

Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMs
by: Kong, Aobo, et al.
Published: (2024)

Towards Spoken Mathematical Reasoning: Benchmarking Speech-based Models over Multi-faceted Math Problems
by: Wei, Chengwei, et al.
Published: (2025)

Self-Play Preference Optimization for Language Model Alignment
by: Wu, Yue, et al.
Published: (2024)

TeXBLEU: Automatic Metric for Evaluate LaTeX Format
by: Jung, Kyudan, et al.
Published: (2024)

SGPO: Self-Generated Preference Optimization based on Self-Improver
by: Lee, Hyeonji, et al.
Published: (2025)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
by: Sun, Lin, et al.
Published: (2025)

Plug-and-Play Training Framework for Preference Optimization
by: Ma, Jingyuan, et al.
Published: (2024)

CoinMath: Harnessing the Power of Coding Instruction for Math LLMs
by: Wei, Chengwei, et al.
Published: (2024)

Talk Less, Call Right: Enhancing Role-Play LLM Agents with Automatic Prompt Optimization and Role Prompting
by: Ruangtanusak, Saksorn, et al.
Published: (2025)

Better Zero-Shot Reasoning with Role-Play Prompting
by: Kong, Aobo, et al.
Published: (2023)

On the Role of Discreteness in Diffusion LLMs
by: Jin, Ziqi, et al.
Published: (2025)

Multi-Agent Tool-Integrated Policy Optimization
by: Mo, Zhanfeng, et al.
Published: (2025)

Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models
by: Shi, Wenhao, et al.
Published: (2024)

Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction
by: Srivastava, Saurabh, et al.
Published: (2025)

Large Language Model-based Role-Playing for Personalized Medical Jargon Extraction
by: Lim, Jung Hoon, et al.
Published: (2024)

InfoDensity: Rewarding Information-Dense Traces for Efficient Reasoning
by: Wei, Chengwei, et al.
Published: (2026)

Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation
by: Ge, Ziyu, et al.
Published: (2025)

PrefPO: Pairwise Preference Prompt Optimization
by: Singhal, Rahul, et al.
Published: (2026)

HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution
by: Hong, Hanhua, et al.
Published: (2026)

Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts
by: Nguyen, Xuan-Phi, et al.
Published: (2023)

Difficulty-Controllable Multiple-Choice Question Generation Using Large Language Models and Direct Preference Optimization
by: Tomikawa, Yuto, et al.
Published: (2025)

MOOSE-Star: Unlocking Tractable Training for Scientific Discovery by Breaking the Complexity Barrier
by: Yang, Zonglin, et al.
Published: (2026)

TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models
by: Ahn, Jaewoo, et al.
Published: (2024)

ORPP: Self-Optimizing Role-playing Prompts to Enhance Language Model Capabilities
by: Duan, Yifan, et al.
Published: (2025)

Interpreting Bias in Large Language Models: A Feature-Based Approach
by: Prakash, Nirmalendu, et al.
Published: (2024)

Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks
by: Kim, Junseok, et al.
Published: (2024)

On the Role of Preference Variance in Preference Optimization
by: Guo, Jiacheng, et al.
Published: (2025)

Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?
by: Chen, Bowen, et al.
Published: (2022)

Revisiting Generalization Across Difficulty Levels: It's Not So Easy
by: Kordi, Yeganeh, et al.
Published: (2025)

DiaTool-DPO: Multi-Turn Direct Preference Optimization for Tool-Augmented Large Language Models
by: Jung, Sunghee, et al.
Published: (2025)

LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
by: Chen, Guanzheng, et al.
Published: (2026)

Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning
by: Tan, Qingyu, et al.
Published: (2023)

Breaking the Pre-Sampling Barrier: Activation-Informed Difficulty-Aware Self-Consistency
by: Yoon, Taewoong, et al.
Published: (2026)

Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisions
by: Hee, Ming Shan, et al.
Published: (2025)

Drift: Decoding-time Personalized Alignments with Implicit User Preferences
by: Kim, Minbeom, et al.
Published: (2025)