Imagen de Portada

RLHFSpec: Breaking the Efficiency Bottleneck in RLHF Training via Adaptive Drafting

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Wang, Siqi, Yang, Hailong, Zhu, Junjie, Wang, Xuezhu, Xu, Yufan, Qian, Depei
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation
por: Cheng, Zelei, et al.
Publicado: (2024)

AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping
por: Dong, Haonan, et al.
Publicado: (2025)

SplaXBERT: Leveraging Mixed Precision Training and Context Splitting for Question Answering
por: Yufan, Zhu, et al.
Publicado: (2024)

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training
por: Xiao, Youshao, et al.
Publicado: (2023)

Breaking the Attention Bottleneck
por: Hilsenbek, Kalle
Publicado: (2024)

Breaking Symmetry Bottlenecks in GNN Readouts
por: Talhi, Mouad, et al.
Publicado: (2026)

Beyond Window-Based Detection: A Graph-Centric Framework for Discrete Log Anomaly Detection
por: Qi, Jiaxing, et al.
Publicado: (2025)

BigMac: Breaking the Pareto Frontier of Compute and Memory in Multimodal LLM Training
por: Zhang, Zili, et al.
Publicado: (2026)

Refining the Information Bottleneck via Adversarial Information Separation
por: Ning, Shuai, et al.
Publicado: (2026)

Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
por: Li, Zongqian, et al.
Publicado: (2026)

Mitigating Reward Over-Optimization in RLHF via Behavior-Supported Regularization
por: Dai, Juntao, et al.
Publicado: (2025)

Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization
por: Wang, Changsheng, et al.
Publicado: (2025)

Efficient Federated RLHF via Zeroth-Order Policy Optimization
por: Wang, Deyi, et al.
Publicado: (2026)

Adaptive Margin RLHF via Preference over Preferences
por: Chittepu, Yaswanth, et al.
Publicado: (2025)

Accelerating RLHF Training with Reward Variance Increase
por: Yang, Zonglin, et al.
Publicado: (2025)

Quantum Machine Learning in Log-based Anomaly Detection: Challenges and Opportunities
por: Qi, Jiaxing, et al.
Publicado: (2024)

Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Models
por: Wang, Haoyu, et al.
Publicado: (2025)

OpenRLHF: An Easy-to-use, Scalable and High-performance RLHF Framework
por: Hu, Jian, et al.
Publicado: (2024)

Exploring Dynamic Properties of Backdoor Training Through Information Bottleneck
por: Liu, Xinyu, et al.
Publicado: (2025)

Policy Optimization in RLHF: The Impact of Out-of-preference Data
por: Li, Ziniu, et al.
Publicado: (2023)

Provably Efficient Online RLHF with One-Pass Reward Modeling
por: Li, Long-Fei, et al.
Publicado: (2025)

Breaking the Bottlenecks: Scalable Diffusion Models for 3D Molecular Generation
por: Das, Adrita, et al.
Publicado: (2026)

Mjolnir: Breaking the Shield of Perturbation-Protected Gradients via Adaptive Diffusion
por: Liu, Xuan, et al.
Publicado: (2024)

Robust Post-Training for Generative Recommenders: Why Exponential Reward-Weighted SFT Outperforms RLHF
por: Chidambaram, Keertana, et al.
Publicado: (2026)

Breaking the Context Bottleneck on Long Time Series Forecasting
por: Ma, Chao, et al.
Publicado: (2024)

Draft, Verify, and Improve: Toward Training-Aware Speculative Decoding
por: Bhansali, Shrenik, et al.
Publicado: (2025)

Mitigating the Alignment Tax of RLHF
por: Lin, Yong, et al.
Publicado: (2023)

RLHF Workflow: From Reward Modeling to Online RLHF
por: Dong, Hanze, et al.
Publicado: (2024)

Towards a Theoretical Understanding to the Generalization of RLHF
por: Li, Zhaochun, et al.
Publicado: (2026)

Factored Causal Representation Learning for Robust Reward Modeling in RLHF
por: Yang, Yupei, et al.
Publicado: (2026)

Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning
por: Zhou, Yang, et al.
Publicado: (2025)

Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
por: Li, Gen, et al.
Publicado: (2025)

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections
por: Xiao, Da, et al.
Publicado: (2025)

SharedRep-RLHF: A Shared Representation Approach to RLHF with Diverse Preferences
por: Mukherjee, Arpan, et al.
Publicado: (2025)

Unifying Stable Optimization and Reference Regularization in RLHF
por: He, Li, et al.
Publicado: (2026)

Learning a Pessimistic Reward Model in RLHF
por: Xu, Yinglun, et al.
Publicado: (2025)

P-EAGLE: Parallel-Drafting EAGLE with Scalable Training
por: Hui, Mude, et al.
Publicado: (2026)

Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
por: Sheng, Jiayuan, et al.
Publicado: (2025)

Controllable Concept Bottleneck Models
por: Lin, Hongbin, et al.
Publicado: (2026)

Breaking the Simplification Bottleneck in Amortized Neural Symbolic Regression
por: Saegert, Paul, et al.
Publicado: (2026)