More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913914908835840 |
|---|---|
| author | Xiong, Weimin Wang, Ke Song, Yifan Liu, Hanchao Zhou, Sai Peng, Wei Li, Sujian |
| author_facet | Xiong, Weimin Wang, Ke Song, Yifan Liu, Hanchao Zhou, Sai Peng, Wei Li, Sujian |
| contents | Current evaluations of tool-integrated LLM agents typically focus on end-to-end tool-usage evaluation while neglecting their stability. This limits their real-world applicability, as various internal or external factors can cause agents to crash or behave abnormally. Our research addresses this by investigating whether agents are vulnerable to errors throughout the entire tool invocation process, including reading tool documentation, selecting tools and generating parameters, and processing the tool's response. Through extensive experiments, we observe that agents are highly susceptible to errors at each stage and agents based on open-source models are more vulnerable than those based on proprietary models. We also find that increasing the model size does not significantly improve tool invocation reasoning and may make agents more vulnerable to attacks resembling normal user instructions. This highlights the importance of evaluating agent stability and offers valuable insights for future LLM development and evaluation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_21967 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents Xiong, Weimin Wang, Ke Song, Yifan Liu, Hanchao Zhou, Sai Peng, Wei Li, Sujian Computation and Language Machine Learning Current evaluations of tool-integrated LLM agents typically focus on end-to-end tool-usage evaluation while neglecting their stability. This limits their real-world applicability, as various internal or external factors can cause agents to crash or behave abnormally. Our research addresses this by investigating whether agents are vulnerable to errors throughout the entire tool invocation process, including reading tool documentation, selecting tools and generating parameters, and processing the tool's response. Through extensive experiments, we observe that agents are highly susceptible to errors at each stage and agents based on open-source models are more vulnerable than those based on proprietary models. We also find that increasing the model size does not significantly improve tool invocation reasoning and may make agents more vulnerable to attacks resembling normal user instructions. This highlights the importance of evaluating agent stability and offers valuable insights for future LLM development and evaluation. |
| title | More Vulnerable than You Think: On the Stability of Tool-Integrated LLM Agents |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2506.21967 |