Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908908149276672 |
|---|---|
| author | Li, Zeping Wang, Hongru Zhao, Yiwen Chen, Guanhua Li, Yixia Chen, Keyang Cao, Yixin Ye, Guangnan Chai, Hongfeng Yin, Zhenfei |
| author_facet | Li, Zeping Wang, Hongru Zhao, Yiwen Chen, Guanhua Li, Yixia Chen, Keyang Cao, Yixin Ye, Guangnan Chai, Hongfeng Yin, Zhenfei |
| contents | Tool-using agents based on Large Language Models (LLMs) excel in tasks such as mathematical reasoning and multi-hop question answering. However, in long trajectories, agents often trigger excessive and low-quality tool calls, increasing latency and degrading inference performance, making managing tool-use behavior challenging. In this work, we conduct entropy-based pilot experiments and observe a strong positive correlation between entropy reduction and high-quality tool calls. Building on this finding, we propose using entropy reduction as a supervisory signal and design two reward strategies to address the differing needs of optimizing tool-use behavior. Sparse outcome rewards provide coarse, trajectory-level guidance to improve efficiency, while dense process rewards offer fine-grained supervision to enhance performance. Experiments across diverse domains show that both reward designs improve tool-use behavior: the former reduces tool calls by 72.07% compared to the average of baselines, while the latter improves performance by 22.27%. These results position entropy reduction as a key mechanism for enhancing tool-use behavior, enabling agents to be more adaptive in real-world applications. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02050 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents Li, Zeping Wang, Hongru Zhao, Yiwen Chen, Guanhua Li, Yixia Chen, Keyang Cao, Yixin Ye, Guangnan Chai, Hongfeng Yin, Zhenfei Artificial Intelligence Software Engineering Tool-using agents based on Large Language Models (LLMs) excel in tasks such as mathematical reasoning and multi-hop question answering. However, in long trajectories, agents often trigger excessive and low-quality tool calls, increasing latency and degrading inference performance, making managing tool-use behavior challenging. In this work, we conduct entropy-based pilot experiments and observe a strong positive correlation between entropy reduction and high-quality tool calls. Building on this finding, we propose using entropy reduction as a supervisory signal and design two reward strategies to address the differing needs of optimizing tool-use behavior. Sparse outcome rewards provide coarse, trajectory-level guidance to improve efficiency, while dense process rewards offer fine-grained supervision to enhance performance. Experiments across diverse domains show that both reward designs improve tool-use behavior: the former reduces tool calls by 72.07% compared to the average of baselines, while the latter improves performance by 22.27%. These results position entropy reduction as a key mechanism for enhancing tool-use behavior, enabling agents to be more adaptive in real-world applications. |
| title | Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents |
| topic | Artificial Intelligence Software Engineering |
| url | https://arxiv.org/abs/2602.02050 |