Cover Image

Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Xiang, Violet, Blagden, Chase, Rafailov, Rafael, Lile, Nathan, Truong, Sang, Finn, Chelsea, Haber, Nick
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
by: Xiang, Violet, et al.
Published: (2025)

Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
by: Albalak, Alon, et al.
Published: (2025)

LitBench: A Benchmark and Dataset for Reliable Evaluation of Creative Writing
by: Fein, Daniel, et al.
Published: (2025)

Generative Reward Models
by: Mahan, Dakota, et al.
Published: (2024)

PERSONA: A Reproducible Testbed for Pluralistic Alignment
by: Castricato, Louis, et al.
Published: (2024)

Disentangling Length from Quality in Direct Preference Optimization
by: Park, Ryan, et al.
Published: (2024)

Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
by: Putta, Pranav, et al.
Published: (2024)

ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code
by: Hua, Tianyu, et al.
Published: (2025)

Leash: Adaptive Length Penalty and Reward Shaping for Efficient Large Reasoning Model
by: Li, Yanhao, et al.
Published: (2025)

MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learning
by: Rafailov, Rafael, et al.
Published: (2024)

Efficient Imitation Learning with Conservative World Models
by: Kolev, Victor, et al.
Published: (2024)

Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
by: Sharma, Aman, et al.
Published: (2025)

Contrastive Preference Learning: Learning from Human Feedback without RL
by: Hejna, Joey, et al.
Published: (2023)

Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
by: Cross, Logan, et al.
Published: (2024)

Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty
by: Yu, Zewei, et al.
Published: (2026)

One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Models
by: Fein, Daniel, et al.
Published: (2026)

Direct Preference Optimization: Your Language Model is Secretly a Reward Model
by: Rafailov, Rafael, et al.
Published: (2023)

When Can Model-Free Reinforcement Learning be Enough for Thinking?
by: Hanna, Josiah P., et al.
Published: (2025)

From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function
by: Rafailov, Rafael, et al.
Published: (2024)

What Matters for Batch Online Reinforcement Learning in Robotics?
by: Dong, Perry, et al.
Published: (2025)

EXPO: Stable Reinforcement Learning with Expressive Policies
by: Dong, Perry, et al.
Published: (2025)

EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models
by: Dong, Perry, et al.
Published: (2026)

Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval
by: Hsu, Sheryl, et al.
Published: (2024)

Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
by: Rafailov, Rafael, et al.
Published: (2024)

Reinforcement Learning via Implicit Imitation Guidance
by: Dong, Perry, et al.
Published: (2025)

Omni-AutoThink: Adaptive Multimodal Reasoning via Reinforcement Learning
by: Yang, Dongchao, et al.
Published: (2025)

Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
by: Liu, Wei, et al.
Published: (2025)

Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
by: Xu, Yuanda, et al.
Published: (2026)

Shorten After You're Right: Lazy Length Penalties for Reasoning RL
by: Yuan, Danlong, et al.
Published: (2025)

Learning to Decide with Just Enough: Information-Theoretic Context Summarization for CMDPs
by: Liu, Peidong, et al.
Published: (2025)

Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
by: Zhou, Yiyang, et al.
Published: (2024)

Polychromic Objectives for Reinforcement Learning
by: Hamid, Jubayer Ibn, et al.
Published: (2025)

The Sound of Syntax: Finetuning and Comprehensive Evaluation of Language Models for Speech Pathology
by: Patel, Fagun, et al.
Published: (2025)

When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning
by: Zhang, Xiaoyun, et al.
Published: (2025)

Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoning
by: Zhou, Zhaomeng, et al.
Published: (2026)

How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning
by: Zhai, Zhiyuan, et al.
Published: (2026)

Affordance-Guided Reinforcement Learning via Visual Prompting
by: Lee, Olivia Y., et al.
Published: (2024)

Adaptive Dual Reasoner: Large Reasoning Models Can Think Efficiently by Hybrid Reasoning
by: Zhang, Yujian, et al.
Published: (2025)

Adaptive Overclocking: Dynamic Control of Thinking Path Length via Real-Time Reasoning Signals
by: Jiang, Shuhao, et al.
Published: (2025)

InftyThink+: Effective and Efficient Infinite-Horizon Reasoning via Reinforcement Learning
by: Yan, Yuchen, et al.
Published: (2026)