Look Before You Leap: Autonomous Exploration for LLM Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866918504408547328 |
|---|---|
| author | Ye, Ziang Shi, Wentao Liu, Yuxin Wang, Yu Cai, Zhengzhou Shi, Yaorui Gu, Qi Cai, Xunliang Feng, Fuli |
| author_facet | Ye, Ziang Shi, Wentao Liu, Yuxin Wang, Yu Cai, Zhengzhou Shi, Yaorui Gu, Qi Cai, Xunliang Feng, Fuli |
| contents | Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_16143 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Look Before You Leap: Autonomous Exploration for LLM Agents Ye, Ziang Shi, Wentao Liu, Yuxin Wang, Yu Cai, Zhengzhou Shi, Yaorui Gu, Qi Cai, Xunliang Feng, Fuli Artificial Intelligence Computation and Language Large language model based agents often fail in unfamiliar environments due to premature exploitation: a tendency to act on prior knowledge before acquiring sufficient environment-specific information. We identify autonomous exploration as a critical yet underexplored capability for building adaptive agents. To formalize and quantify this capability, we introduce Exploration Checkpoint Coverage, a verifiable metric that measures how broadly an agent discovers key states, objects, and affordances. Our systematic evaluation reveals that agents trained with standard task-oriented reinforcement learning consistently exhibit narrow and repetitive behaviors that impede downstream performance. To address this limitation, we develop a training strategy that interleaves task-execution rollouts and exploration rollouts, with each type of rollout optimized by its corresponding verifiable reward. Building on this training strategy, we propose the Explore-then-Act paradigm, which decouples information-gathering from task execution: agents first utilize an interaction budget to acquire grounded environmental knowledge, then leverage it for task resolution. Our results demonstrate that learning to systematically explore is imperative for building generalizable and real-world-ready agents. |
| title | Look Before You Leap: Autonomous Exploration for LLM Agents |
| topic | Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2605.16143 |