Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866908808100446208 |
|---|---|
| author | Zheng, Jingnan Luo, Yanzhen Xu, Jingjun Liu, Bingnan Chen, Yuxin Cui, Chenhang Deng, Gelei Lu, Chaochao Wang, Xiang Zhang, An Chua, Tat-Seng |
| author_facet | Zheng, Jingnan Luo, Yanzhen Xu, Jingjun Liu, Bingnan Chen, Yuxin Cui, Chenhang Deng, Gelei Lu, Chaochao Wang, Xiang Zhang, An Chua, Tat-Seng |
| contents | Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm. Existing agent safety evaluations rely on risk-oriented tasks tailored to specific agent settings, resulting in limited coverage of safety risk space and failing to assess agent safety behavior during long-horizon, interactive task execution in complex real-world deployments. Moreover, their specialization to particular agent settings limits adaptability across diverse agent configurations. To address these limitations, we propose Risky-Bench, a framework that enables systematic agent safety evaluation grounded in real-world deployment. Risky-Bench organizes evaluation around domain-agnostic safety principles to derive context-aware safety rubrics that delineate safety space, and systematically evaluates safety risks across this space through realistic task execution under varying threat assumptions. When applied to life-assist agent settings, Risky-Bench uncovers substantial safety risks in state-of-the-art agents under realistic execution conditions. Moreover, as a well-structured evaluation pipeline, Risky-Bench is not confined to life-assist scenarios and can be adapted to other deployment settings to construct environment-specific safety evaluations, providing an extensible methodology for agent safety assessment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_03100 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment Zheng, Jingnan Luo, Yanzhen Xu, Jingjun Liu, Bingnan Chen, Yuxin Cui, Chenhang Deng, Gelei Lu, Chaochao Wang, Xiang Zhang, An Chua, Tat-Seng Artificial Intelligence Large Language Models (LLMs) are increasingly deployed as agents that operate in real-world environments, introducing safety risks beyond linguistic harm. Existing agent safety evaluations rely on risk-oriented tasks tailored to specific agent settings, resulting in limited coverage of safety risk space and failing to assess agent safety behavior during long-horizon, interactive task execution in complex real-world deployments. Moreover, their specialization to particular agent settings limits adaptability across diverse agent configurations. To address these limitations, we propose Risky-Bench, a framework that enables systematic agent safety evaluation grounded in real-world deployment. Risky-Bench organizes evaluation around domain-agnostic safety principles to derive context-aware safety rubrics that delineate safety space, and systematically evaluates safety risks across this space through realistic task execution under varying threat assumptions. When applied to life-assist agent settings, Risky-Bench uncovers substantial safety risks in state-of-the-art agents under realistic execution conditions. Moreover, as a well-structured evaluation pipeline, Risky-Bench is not confined to life-assist scenarios and can be adapted to other deployment settings to construct environment-specific safety evaluations, providing an extensible methodology for agent safety assessment. |
| title | Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2602.03100 |