Imagen de Portada

Finding the Sweet Spot: Preference Data Construction for Scaling Preference Optimization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Xiao, Yao, Ye, Hai, Chen, Linyao, Ng, Hwee Tou, Bing, Lidong, Li, Xiaoli, Lee, Roy Ka-wei
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Revisiting Self-Play Preference Optimization: On the Role of Prompt Difficulty
por: Xiao, Yao, et al.
Publicado: (2025)

Preference-Guided Reflective Sampling for Aligning Language Models
por: Ye, Hai, et al.
Publicado: (2024)

Towards Robust Temporal Reasoning of Large Language Models via a Multi-Hop QA Dataset and Pseudo-Instruction Tuning
por: Tan, Qingyu, et al.
Publicado: (2023)

Self-Judge: Selective Instruction Following with Alignment Self-Evaluation
por: Ye, Hai, et al.
Publicado: (2024)

Multi-Agent Sampling: Scaling Inference Compute for Data Synthesis with Tree Search-Based Agentic Collaboration
por: Ye, Hai, et al.
Publicado: (2024)

Unlocking Temporal Question Answering for Large Language Models with Tailor-Made Reasoning Logic
por: Li, Xingxuan, et al.
Publicado: (2023)

Document Reconstruction Unlocks Scalable Long-Context RLVR
por: Xiao, Yao, et al.
Publicado: (2026)

Think&Cite: Improving Attributed Text Generation with Self-Guided Tree Search and Progress Reward Modeling
por: Li, Junyi, et al.
Publicado: (2024)

OpenSeal: Good, Fast, and Cheap Construction of an Open-Source Southeast Asian LLM via Parallel Data
por: Nguyen, Tan Sang, et al.
Publicado: (2026)

Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models
por: Li, Junyi, et al.
Publicado: (2025)

DOCE: Finding the Sweet Spot for Execution-Based Code Generation
por: Li, Haau-Sing, et al.
Publicado: (2024)

LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization
por: Chen, Guanzheng, et al.
Publicado: (2025)

Parametric Knowledge is Not All You Need: Toward Honest Large Language Models via Retrieval of Pretraining Data
por: Kusuma, Christopher Adrian, et al.
Publicado: (2026)

Just What You Desire: Constrained Timeline Summarization with Self-Reflection for Enhanced Relevance
por: Qorib, Muhammad Reza, et al.
Publicado: (2024)

Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models
por: Qorib, Muhammad Reza, et al.
Publicado: (2025)

Efficient and Interpretable Grammatical Error Correction with Mixture of Experts
por: Qorib, Muhammad Reza, et al.
Publicado: (2024)

SSL: Sweet Spot Learning for Differentiated Guidance in Agentic Optimization
por: Wu, Jinyang, et al.
Publicado: (2026)

SlideTailor: Personalized Presentation Slide Generation for Scientific Papers
por: Zeng, Wenzheng, et al.
Publicado: (2025)

No Preference Left Behind: Group Distributional Preference Optimization
por: Yao, Binwei, et al.
Publicado: (2024)

DeepResearchEval: An Automated Framework for Deep Research Task Construction and Agentic Evaluation
por: Wang, Yibo, et al.
Publicado: (2026)

Multiplayer Nash Preference Optimization
por: Wu, Fang, et al.
Publicado: (2025)

Stable Preference Optimization: A Bilevel Approach to Catastrophic Preference Shift
por: Jian, Chengtao, et al.
Publicado: (2025)

Game of Thought: Robust Information Seeking with Large Language Models Using Game Theory
por: Cui, Langyuan, et al.
Publicado: (2026)

TSO: Self-Training with Scaled Preference Optimization
por: Chen, Kaihui, et al.
Publicado: (2024)

Diverse Preference Optimization
por: Lanchantin, Jack, et al.
Publicado: (2025)

Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models
por: Shi, Wenhao, et al.
Publicado: (2024)

ToxiCloakCN: Evaluating Robustness of Offensive Language Detection in Chinese with Cloaking Perturbations
por: Xiao, Yunze, et al.
Publicado: (2024)

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings
por: Liu, Tong, et al.
Publicado: (2025)

The CoNLL-2013 Shared Task on Grammatical Error Correction
por: Ng, Hwee Tou, et al.
Publicado: (2025)

Geometric-Averaged Preference Optimization for Soft Preference Labels
por: Furuta, Hiroki, et al.
Publicado: (2024)

Factorized Learning for Temporally Grounded Video-Language Models
por: Zeng, Wenzheng, et al.
Publicado: (2025)

BPO: Revisiting Preference Modeling in Direct Preference Optimization
por: Sun, Lin, et al.
Publicado: (2025)

Hit the Sweet Spot! Span-Level Ensemble for Large Language Models
por: Xu, Yangyifan, et al.
Publicado: (2024)

On the Role of Preference Variance in Preference Optimization
por: Guo, Jiacheng, et al.
Publicado: (2025)

Alignment Data Map for Efficient Preference Data Selection and Diagnosis
por: Lee, Seohyeong, et al.
Publicado: (2025)

MemRerank: Preference Memory for Personalized Product Reranking
por: Peng, Zhiyuan, et al.
Publicado: (2026)

Statistical Rejection Sampling Improves Preference Optimization
por: Liu, Tianqi, et al.
Publicado: (2023)

Preference Optimization for Reasoning with Pseudo Feedback
por: Jiao, Fangkai, et al.
Publicado: (2024)

Multi-Agent Tool-Integrated Policy Optimization
por: Mo, Zhanfeng, et al.
Publicado: (2025)

PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Training
por: Cheng, Yuhan, et al.
Publicado: (2026)