Cover Image

HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languages

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Zhilin, Zeng, Jiaqi, Delalleau, Olivier, Shin, Hoo-Chang, Soares, Felipe, Bukharin, Alexander, Evans, Ellie, Dong, Yi, Kuchaiev, Oleksii
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

HelpSteer2-Preference: Complementing Ratings with Preferences
by: Wang, Zhilin, et al.
Published: (2024)

HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Tasks
by: Wang, Zhilin, et al.
Published: (2025)

HelpSteer2: Open-source dataset for training top-performing reward models
by: Wang, Zhilin, et al.
Published: (2024)

RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards
by: Wang, Zhilin, et al.
Published: (2025)

Reward-aware Preference Optimization: A Unified Mathematical Framework for Model Alignment
by: Sun, Shengyang, et al.
Published: (2025)

Adversarial Training of Reward Models
by: Bukharin, Alexander, et al.
Published: (2025)

Think Twice: Branch-and-Rethink Reasoning Reward Model
by: Jiao, Yizhu, et al.
Published: (2025)

GPT vs RETRO: Exploring the Intersection of Retrieval and Parameter-Efficient Fine-Tuning
by: Ficek, Aleksander, et al.
Published: (2024)

Diverging Preferences: When do Annotators Disagree and do Models Know?
by: Zhang, Michael JQ, et al.
Published: (2024)

How Humans Help LLMs: Assessing and Incentivizing Human Preference Annotators
by: Liu, Shang, et al.
Published: (2025)

NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment
by: Shen, Gerald, et al.
Published: (2024)

Adaptive Preference Scaling for Reinforcement Learning with Human Feedback
by: Hong, Ilgee, et al.
Published: (2024)

Deep Reinforcement Learning from Hierarchical Preference Design
by: Bukharin, Alexander, et al.
Published: (2023)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
by: Kim, Dongyoung, et al.
Published: (2024)

Tied-Lora: Enhancing parameter efficiency of LoRA with weight tying
by: Renduchintala, Adithya, et al.
Published: (2023)

Steerable Chatbots: Personalizing LLMs with Preference-Based Activation Steering
by: Bo, Jessica Y., et al.
Published: (2025)

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment
by: Yan, Ziang, et al.
Published: (2024)

Reinforcement Learning from Diverse Human Preferences
by: Xue, Wanqi, et al.
Published: (2023)

RMGAP: Benchmarking the Generalization of Reward Models across Diverse Preferences
by: Zhou, Yangyang, et al.
Published: (2026)

Tokenization Preference for Human and Machine Learning Model: An Annotation Study
by: Hiraoka, Tatsuya, et al.
Published: (2023)

Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
by: Cao, Guining, et al.
Published: (2026)

MaxMin-RLHF: Alignment with Diverse Human Preferences
by: Chakraborty, Souradip, et al.
Published: (2024)

Diverse Preference Optimization
by: Lanchantin, Jack, et al.
Published: (2025)

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
by: Liang, Ren-Wei, et al.
Published: (2025)

Explaining and Breaking the Safety-Helpfulness Ceiling via Preference Dimensional Expansion
by: Huang, ShiYing, et al.
Published: (2026)

Can Revealed Preferences Clarify LLM Alignment and Steering?
by: Yamin, Khurram, et al.
Published: (2026)

Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training
by: Liu, Mingjie, et al.
Published: (2025)

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation
by: Liang, Chao, et al.
Published: (2025)

Preference Score Distillation: Leveraging 2D Rewards to Align Text-to-3D Generation with Human Preference
by: Leng, Jiaqi, et al.
Published: (2026)

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
by: Chiang, Wei-Lin, et al.
Published: (2024)

Debiasing Online Preference Learning via Preference Feature Preservation
by: Kim, Dongyoung, et al.
Published: (2025)

Quiz Helps Determine Attendees' Round Table Preferences
by: Erin Sandage
Published: (2024)

HPS: Hard Preference Sampling for Human Preference Alignment
by: Zou, Xiandong, et al.
Published: (2025)

MusicRL: Aligning Music Generation to Human Preferences
by: Cideron, Geoffrey, et al.
Published: (2024)

Bayesian Preference Elicitation with Language Models
by: Handa, Kunal, et al.
Published: (2024)

Rethinking Diverse Human Preference Learning through Principal Component Analysis
by: Luo, Feng, et al.
Published: (2025)

Enhancing the Preference Extractor in Multi-turn Dialogues: From Annotating Disasters to Accurate Preference Extraction
by: Wang, Cheng, et al.
Published: (2025)

Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning
by: Yang, Sen, et al.
Published: (2024)

Cross-Modal Content Optimization for Steering Web Agent Preferences
by: Jiang, Tanqiu, et al.
Published: (2025)

Evaluating and Steering Modality Preferences in Multimodal Large Language Model
by: Zhang, Yu, et al.
Published: (2025)