A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910915277881344 |
|---|---|
| author | Huang, Yuting Ding, Leilei Tang, Zhipeng Wang, Tianfu Lin, Xinrui Zhang, Wuyang Ma, Mingxiao Zhang, Yanyong |
| author_facet | Huang, Yuting Ding, Leilei Tang, Zhipeng Wang, Tianfu Lin, Xinrui Zhang, Wuyang Ma, Mingxiao Zhang, Yanyong |
| contents | Large Language Models (LLMs) exhibit substantial promise in enhancing task-planning capabilities within embodied agents due to their advanced reasoning and comprehension. However, the systemic safety of these agents remains an underexplored frontier. In this study, we present Safe-BeAl, an integrated framework for the measurement (SafePlan-Bench) and alignment (Safe-Align) of LLM-based embodied agents' behaviors. SafePlan-Bench establishes a comprehensive benchmark for evaluating task-planning safety, encompassing 2,027 daily tasks and corresponding environments distributed across 8 distinct hazard categories (e.g., Fire Hazard). Our empirical analysis reveals that even in the absence of adversarial inputs or malicious intent, LLM-based agents can exhibit unsafe behaviors. To mitigate these hazards, we propose Safe-Align, a method designed to integrate physical-world safety knowledge into LLM-based embodied agents while maintaining task-specific performance. Experiments across a variety of settings demonstrate that Safe-BeAl provides comprehensive safety validation, improving safety by 8.55 - 15.22%, compared to embodied agents based on GPT-4, while ensuring successful task completion. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_14650 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents Huang, Yuting Ding, Leilei Tang, Zhipeng Wang, Tianfu Lin, Xinrui Zhang, Wuyang Ma, Mingxiao Zhang, Yanyong Artificial Intelligence Large Language Models (LLMs) exhibit substantial promise in enhancing task-planning capabilities within embodied agents due to their advanced reasoning and comprehension. However, the systemic safety of these agents remains an underexplored frontier. In this study, we present Safe-BeAl, an integrated framework for the measurement (SafePlan-Bench) and alignment (Safe-Align) of LLM-based embodied agents' behaviors. SafePlan-Bench establishes a comprehensive benchmark for evaluating task-planning safety, encompassing 2,027 daily tasks and corresponding environments distributed across 8 distinct hazard categories (e.g., Fire Hazard). Our empirical analysis reveals that even in the absence of adversarial inputs or malicious intent, LLM-based agents can exhibit unsafe behaviors. To mitigate these hazards, we propose Safe-Align, a method designed to integrate physical-world safety knowledge into LLM-based embodied agents while maintaining task-specific performance. Experiments across a variety of settings demonstrate that Safe-BeAl provides comprehensive safety validation, improving safety by 8.55 - 15.22%, compared to embodied agents based on GPT-4, while ensuring successful task completion. |
| title | A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agents |
| topic | Artificial Intelligence |
| url | https://arxiv.org/abs/2504.14650 |