ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866913468909617152 |
|---|---|
| author | Ye, Junjie Li, Sixian Li, Guanyu Huang, Caishuang Gao, Songyang Wu, Yilong Zhang, Qi Gui, Tao Huang, Xuanjing |
| author_facet | Ye, Junjie Li, Sixian Li, Guanyu Huang, Caishuang Gao, Songyang Wu, Yilong Zhang, Qi Gui, Tao Huang, Xuanjing |
| contents | Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios. While current research primarily emphasizes leveraging tools to augment LLMs, it frequently neglects emerging safety considerations tied to their application. To fill this gap, we present *ToolSword*, a comprehensive framework dedicated to meticulously investigating safety issues linked to LLMs in tool learning. Specifically, ToolSword delineates six safety scenarios for LLMs in tool learning, encompassing **malicious queries** and **jailbreak attacks** in the input stage, **noisy misdirection** and **risky cues** in the execution stage, and **harmful feedback** and **error conflicts** in the output stage. Experiments conducted on 11 open-source and closed-source LLMs reveal enduring safety challenges in tool learning, such as handling harmful queries, employing risky tools, and delivering detrimental feedback, which even GPT-4 is susceptible to. Moreover, we conduct further studies with the aim of fostering research on tool learning safety. The data is released in https://github.com/Junjie-Ye/ToolSword. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10753 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages Ye, Junjie Li, Sixian Li, Guanyu Huang, Caishuang Gao, Songyang Wu, Yilong Zhang, Qi Gui, Tao Huang, Xuanjing Computation and Language Artificial Intelligence Tool learning is widely acknowledged as a foundational approach or deploying large language models (LLMs) in real-world scenarios. While current research primarily emphasizes leveraging tools to augment LLMs, it frequently neglects emerging safety considerations tied to their application. To fill this gap, we present *ToolSword*, a comprehensive framework dedicated to meticulously investigating safety issues linked to LLMs in tool learning. Specifically, ToolSword delineates six safety scenarios for LLMs in tool learning, encompassing **malicious queries** and **jailbreak attacks** in the input stage, **noisy misdirection** and **risky cues** in the execution stage, and **harmful feedback** and **error conflicts** in the output stage. Experiments conducted on 11 open-source and closed-source LLMs reveal enduring safety challenges in tool learning, such as handling harmful queries, employing risky tools, and delivering detrimental feedback, which even GPT-4 is susceptible to. Moreover, we conduct further studies with the aim of fostering research on tool learning safety. The data is released in https://github.com/Junjie-Ye/ToolSword. |
| title | ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stages |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2402.10753 |