SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908984717344768 |
|---|---|
| author | Torres-Fonseca, Josue Deng, Naihao Dai, Yinpei Storks, Shane Zhang, Yichi Mihalcea, Rada Kennington, Casey Chai, Joyce |
| author_facet | Torres-Fonseca, Josue Deng, Naihao Dai, Yinpei Storks, Shane Zhang, Yichi Mihalcea, Rada Kennington, Casey Chai, Joyce |
| contents | Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient. We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards. While existing safety evaluations focus on hazard recognition through disembodied question answering (QA) settings, we evaluate eleven state-of-the-art models from the Qwen, Gemma, and Gemini families on not only hazard recognition, but also active risk mitigation through embodied planning. Our experimental results reveal a significant alignment gap: while models can accurately recognize hazards in QA settings, average mitigation success rates for these hazards are low in comparison. Our findings demonstrate that static evaluations through QA are insufficient for physical safety, thus we advocate for a paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts. We open-source our code and dataset under https://github.com/sled-group/SafetyALFRED.git |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19638 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models Torres-Fonseca, Josue Deng, Naihao Dai, Yinpei Storks, Shane Zhang, Yichi Mihalcea, Rada Kennington, Casey Chai, Joyce Artificial Intelligence Computation and Language Robotics Multimodal Large Language Models are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient. We introduce SafetyALFRED, built upon the embodied agent benchmark ALFRED, augmented with six categories of real-world kitchen hazards. While existing safety evaluations focus on hazard recognition through disembodied question answering (QA) settings, we evaluate eleven state-of-the-art models from the Qwen, Gemma, and Gemini families on not only hazard recognition, but also active risk mitigation through embodied planning. Our experimental results reveal a significant alignment gap: while models can accurately recognize hazards in QA settings, average mitigation success rates for these hazards are low in comparison. Our findings demonstrate that static evaluations through QA are insufficient for physical safety, thus we advocate for a paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts. We open-source our code and dataset under https://github.com/sled-group/SafetyALFRED.git |
| title | SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models |
| topic | Artificial Intelligence Computation and Language Robotics |
| url | https://arxiv.org/abs/2604.19638 |