Cover Image

Does RLHF Scale? Exploring the Impacts From Data, Model, and Method

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Hou, Zhenyu, Du, Pengfan, Niu, Yilin, Du, Zhengxiao, Zeng, Aohan, Liu, Xiao, Huang, Minlie, Wang, Hongning, Tang, Jie, Dong, Yuxiao
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback
by: Hou, Zhenyu, et al.
Published: (2024)

Understanding Emergent Abilities of Language Models from the Loss Perspective
by: Du, Zhengxiao, et al.
Published: (2024)

Scaling Speech-Text Pre-training with Synthetic Interleaved Data
by: Zeng, Aohan, et al.
Published: (2024)

GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot
by: Zeng, Aohan, et al.
Published: (2024)

Data-Efficient RLVR via Off-Policy Influence Guidance
by: Zhu, Erle, et al.
Published: (2025)

ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
by: Xu, Yifan, et al.
Published: (2024)

IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation
by: Wen, Bosi, et al.
Published: (2025)

CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation
by: Ke, Pei, et al.
Published: (2023)

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
by: Bai, Yushi, et al.
Published: (2023)

IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse
by: Bai, Yushi, et al.
Published: (2026)

Black-Box Prompt Optimization: Aligning Large Language Models without Model Training
by: Cheng, Jiale, et al.
Published: (2023)

Towards Efficient Exact Optimization of Language Model Alignment
by: Ji, Haozhe, et al.
Published: (2024)

AutoDetect: Towards a Unified Framework for Automated Weakness Detection in Large Language Models
by: Cheng, Jiale, et al.
Published: (2024)

LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
by: Gui, Jiayi, et al.
Published: (2024)

SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling
by: Wang, Haoran, et al.
Published: (2025)

APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding
by: Liu, Mingdao, et al.
Published: (2024)

SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models
by: Cheng, Jiale, et al.
Published: (2024)

ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario
by: Zhong, Lucen, et al.
Published: (2025)

Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning
by: Wen, Jiaxin, et al.
Published: (2024)

LongSafety: Evaluating Long-Context Safety of Large Language Models
by: Lu, Yida, et al.
Published: (2025)

IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation
by: Wen, Bosi, et al.
Published: (2026)

Data Selection via Optimal Control for Language Models
by: Gu, Yuxian, et al.
Published: (2024)

Language Model Decoding as Direct Metrics Optimization
by: Ji, Haozhe, et al.
Published: (2023)

GraphAlign: Pretraining One Graph Neural Network on Multiple Graphs via Feature Alignment
by: Hou, Zhenyu, et al.
Published: (2024)

VPO: Aligning Text-to-Video Generation Models with Prompt Optimization
by: Cheng, Jiale, et al.
Published: (2025)

AgentBench: Evaluating LLMs as Agents
by: Liu, Xiao, et al.
Published: (2023)

SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Models
by: Zhang, Dan, et al.
Published: (2024)

AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework
by: Zhang, Hanchen, et al.
Published: (2025)

TreeRL: LLM Reinforcement Learning with On-Policy Tree Search
by: Hou, Zhenyu, et al.
Published: (2025)

HoWToBench: Holistic Evaluation for LLM's Capability in Human-level Writing using Tree of Writing
by: Feng, Andrew Zhuoer, et al.
Published: (2026)

Trust-Region Adaptive Policy Optimization
by: Su, Mingyu, et al.
Published: (2025)

Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints
by: Yang, Junxiao, et al.
Published: (2025)

LongReward: Improving Long-context Large Language Models with AI Feedback
by: Zhang, Jiajie, et al.
Published: (2024)

T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
by: Hou, Zhenyu, et al.
Published: (2025)

Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!
by: Zhang, Zhexin, et al.
Published: (2025)

RLHF Workflow: From Reward Modeling to Online RLHF
by: Dong, Hanze, et al.
Published: (2024)

Think Socially via Cognitive Reasoning
by: Zhou, Jinfeng, et al.
Published: (2025)

Language Models Learn to Mislead Humans via RLHF
by: Wen, Jiaxin, et al.
Published: (2024)

Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization
by: Zhang, Zhexin, et al.
Published: (2023)

Learning Task Decomposition to Assist Humans in Competitive Programming
by: Wen, Jiaxin, et al.
Published: (2024)