Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911200861749248 |
|---|---|
| author | Hao, Bingguang Xu, Zengzhuang Wang, Maolin Wen, Yuntao Chen, Yicheng Peng, Cunyin Chen, Long Wang, Dong Zhao, Xiangyu Gu, Jinjie Zhuang, Chenyi Zhang, Ji |
| author_facet | Hao, Bingguang Xu, Zengzhuang Wang, Maolin Wen, Yuntao Chen, Yicheng Peng, Cunyin Chen, Long Wang, Dong Zhao, Xiangyu Gu, Jinjie Zhuang, Chenyi Zhang, Ji |
| contents | The effective training of Large Language Models (LLMs) for function calling faces a critical challenge: balancing exploration of complex reasoning paths with stable policy optimization. Standard methods like Supervised Fine-Tuning (SFT) fail to instill robust reasoning, and traditional Reinforcement Learning (RL) struggles with inefficient exploration. We propose \textbf{EGPO}, a new RL framework built upon Group Relative Policy Optimization (GRPO), designed to address this challenge directly. The core of EGPO is an entropy-enhanced advantage function that integrates the entropy of the model's Chain-of-Thought (CoT) into the policy gradient computation. This encourages the generation of diverse reasoning strategies. To maintain optimization direction, the entropy bonus is carefully constrained by a clipping mechanism. Complemented by a strict, binary reward signal, EGPO effectively guides the model towards discovering structured and accurate tool invocation patterns. On the challenging Berkeley Function Calling Leaderboard (BFCL), a 4B-parameter model trained with EGPO sets a new state-of-the-art among models of comparable size, surpassing a range of strong competitors, including GPT-4o and Gemini-2.5. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_05118 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling Hao, Bingguang Xu, Zengzhuang Wang, Maolin Wen, Yuntao Chen, Yicheng Peng, Cunyin Chen, Long Wang, Dong Zhao, Xiangyu Gu, Jinjie Zhuang, Chenyi Zhang, Ji Machine Learning Artificial Intelligence Computation and Language The effective training of Large Language Models (LLMs) for function calling faces a critical challenge: balancing exploration of complex reasoning paths with stable policy optimization. Standard methods like Supervised Fine-Tuning (SFT) fail to instill robust reasoning, and traditional Reinforcement Learning (RL) struggles with inefficient exploration. We propose \textbf{EGPO}, a new RL framework built upon Group Relative Policy Optimization (GRPO), designed to address this challenge directly. The core of EGPO is an entropy-enhanced advantage function that integrates the entropy of the model's Chain-of-Thought (CoT) into the policy gradient computation. This encourages the generation of diverse reasoning strategies. To maintain optimization direction, the entropy bonus is carefully constrained by a clipping mechanism. Complemented by a strict, binary reward signal, EGPO effectively guides the model towards discovering structured and accurate tool invocation patterns. On the challenging Berkeley Function Calling Leaderboard (BFCL), a 4B-parameter model trained with EGPO sets a new state-of-the-art among models of comparable size, surpassing a range of strong competitors, including GPT-4o and Gemini-2.5. |
| title | Reasoning through Exploration: A Reinforcement Learning Framework for Robust Function Calling |
| topic | Machine Learning Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2508.05118 |