Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zeng, Zhiyuan, Cheng, Qinyuan, Yin, Zhangyue, Wang, Bo, Li, Shimin, Zhou, Yunhua, Guo, Qipeng, Huang, Xuanjing, Qiu, Xipeng
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?
by: Zeng, Zhiyuan, et al.
Published: (2025)

ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models
by: Yin, Zhangyue, et al.
Published: (2025)

Dynamic and Generalizable Process Reward Modeling
by: Yin, Zhangyue, et al.
Published: (2025)

How to Set the Learning Rate for Large-Scale Pre-training?
by: Zhou, Yunhua, et al.
Published: (2026)

RLoop: An Self-Improving Framework for Reinforcement Learning with Iterative Policy Initialization
by: Zhiyuan, Zeng, et al.
Published: (2025)

Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections
by: Wang, Bo, et al.
Published: (2025)

Turn Waste into Worth: Rectifying Top-$k$ Router of MoE
by: Zeng, Zhiyuan, et al.
Published: (2024)

Error Classification of Large Language Models on Math Word Problems: A Dynamically Adaptive Framework
by: Sun, Yuhong, et al.
Published: (2025)

R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning
by: Li, Yuan, et al.
Published: (2025)

How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective
by: Peng, Runyu, et al.
Published: (2026)

Agent Alignment in Evolving Social Norms
by: Li, Shimin, et al.
Published: (2024)

Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models
by: Yin, Zhangyue, et al.
Published: (2024)

Scaling Laws for Fact Memorization of Large Language Models
by: Lu, Xingyu, et al.
Published: (2024)

Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem
by: Sun, Yuhong, et al.
Published: (2024)

Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model
by: Wang, Siyin, et al.
Published: (2024)

VisuoThink: Empowering LVLM Reasoning with Multimodal Tree Search
by: Wang, Yikun, et al.
Published: (2025)

BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning
by: Li, Yuan, et al.
Published: (2026)

How to Mitigate Overfitting in Weak-to-strong Generalization?
by: Shi, Junhao, et al.
Published: (2025)

Can AI Assistants Know What They Don't Know?
by: Cheng, Qinyuan, et al.
Published: (2024)

Unified Active Retrieval for Retrieval Augmented Generation
by: Cheng, Qinyuan, et al.
Published: (2024)

Explicit Multi-head Attention for Inter-head Interaction in Large Language Models
by: Peng, Runyu, et al.
Published: (2026)

LLM can Achieve Self-Regulation via Hyperparameter Aware Generation
by: Wang, Siyin, et al.
Published: (2024)

Data-free Weight Compress and Denoise for Large Language Models
by: Peng, Runyu, et al.
Published: (2024)

Case2Code: Scalable Synthetic Data for Code Generation
by: Shao, Yunfan, et al.
Published: (2024)

Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioning
by: Zhang, Yechen, et al.
Published: (2026)

AdaLomo: Low-memory Optimization with Adaptive Learning Rate
by: Lv, Kai, et al.
Published: (2023)

WESR: Scaling and Evaluating Word-level Event-Speech Recognition
by: Yang, Chenchen, et al.
Published: (2026)

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders
by: He, Zhengfu, et al.
Published: (2024)

How to Set the Batch Size for Large-Scale Pre-training?
by: Zhou, Yunhua, et al.
Published: (2026)

DuoDecoding: Hardware-aware Heterogeneous Speculative Decoding with Dynamic Multi-Sequence Drafting
by: Lv, Kai, et al.
Published: (2025)

FamilyTool: A Multi-hop Personalized Tool Use Benchmark
by: Wang, Yuxin, et al.
Published: (2025)

VehicleWorld: A Highly Integrated Multi-Device Environment for Intelligent Vehicle Interaction
by: Yang, Jie, et al.
Published: (2025)

Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
by: Fei, Zhaoye, et al.
Published: (2024)

ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning
by: Zhu, Changtai, et al.
Published: (2025)

Can Language Models Learn to Skip Steps?
by: Liu, Tengxiao, et al.
Published: (2024)

AdaptR1: Reinforcement Learning Based Adaptive Interleaved Thinking in Multi-hop Question Answering
by: Wang, Yuxin, et al.
Published: (2026)

AutoLogi: Automated Generation of Logic Puzzles for Evaluating Reasoning Abilities of Large Language Models
by: Zhu, Qin, et al.
Published: (2025)

Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
by: He, Zhengfu, et al.
Published: (2024)

DenoSent: A Denoising Objective for Self-Supervised Sentence Representation Learning
by: Wang, Xinghao, et al.
Published: (2024)

CodecBench: A Comprehensive Benchmark for Acoustic and Semantic Evaluation
by: Deng, Ruifan, et al.
Published: (2025)