Can RL Improve Generalization of LLM Agents? An Empirical Study
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911509060255744 |
|---|---|
| author | Xi, Zhiheng Guo, Xin Liu, Jiaqi Zhang, Jiazheng Fan, Yutao Zhang, Zhihao Liu, Shichun Chai, Mingxu Shi, Xiaowei Zhai, Yitao Cai, Xunliang Gui, Tao Zhang, Qi Huang, Xuanjing |
| author_facet | Xi, Zhiheng Guo, Xin Liu, Jiaqi Zhang, Jiazheng Fan, Yutao Zhang, Zhihao Liu, Shichun Chai, Mingxu Shi, Xiaowei Zhai, Yitao Cai, Xunliang Gui, Tao Zhang, Qi Huang, Xuanjing |
| contents | Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback. However, most existing evaluations remain largely in-domain: training and testing are conducted in the same environment or even on the same tasks. In real-world deployment, agents may operate in unseen environments with different background knowledge, observation spaces, and action interfaces. To characterize the generalization profile of RFT under such shifts, we conduct a systematic study along three axes: (1) within-environment generalization across task difficulty, (2) cross-environment transfer to unseen environments, and (3) sequential multi-environment training to quantify transfer and forgetting. Our results show that RFT generalizes well across task difficulty within an environment, but exhibits weaker transfer to unseen environments, which correlates with shifts in both semantic priors and observation/action interfaces. In contrast, sequential training yields promising downstream gains with minimal upstream forgetting, and mixture training across environments improves the overall balance. We further provide detailed analyses and deeper insights, and hope our work helps the community develop and deploy generalizable LLM agents. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_12011 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Can RL Improve Generalization of LLM Agents? An Empirical Study Xi, Zhiheng Guo, Xin Liu, Jiaqi Zhang, Jiazheng Fan, Yutao Zhang, Zhihao Liu, Shichun Chai, Mingxu Shi, Xiaowei Zhai, Yitao Cai, Xunliang Gui, Tao Zhang, Qi Huang, Xuanjing Artificial Intelligence Reinforcement fine-tuning (RFT) has shown promise for training LLM agents to perform multi-turn decision-making based on environment feedback. However, most existing evaluations remain largely in-domain: training and testing are conducted in the same environment or even on the same tasks. In real-world deployment, agents may operate in unseen environments with different background knowledge, observation spaces, and action interfaces. To characterize the generalization profile of RFT under such shifts, we conduct a systematic study along three axes: (1) within-environment generalization across task difficulty, (2) cross-environment transfer to unseen environments, and (3) sequential multi-environment training to quantify transfer and forgetting. Our results show that RFT generalizes well across task difficulty within an environment, but exhibits weaker transfer to unseen environments, which correlates with shifts in both semantic priors and observation/action interfaces. In contrast, sequential training yields promising downstream gains with minimal upstream forgetting, and mixture training across environments improves the overall balance. We further provide detailed analyses and deeper insights, and hope our work helps the community develop and deploy generalizable LLM agents. |
| title | Can RL Improve Generalization of LLM Agents? An Empirical Study |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2603.12011 |