Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914023441694720 |
|---|---|
| author | Ma, Xinbei Wang, Yiting Yao, Yao Yuan, Tongxin Zhang, Aston Zhang, Zhuosheng Zhao, Hai |
| author_facet | Ma, Xinbei Wang, Yiting Yao, Yao Yuan, Tongxin Zhang, Aston Zhang, Zhuosheng Zhao, Hai |
| contents | This paper investigates the faithfulness of multimodal large language model (MLLM) agents in a graphical user interface (GUI) environment, aiming to address the research question of whether multimodal GUI agents can be distracted by environmental context. A general scenario is proposed where both the user and the agent are benign, and the environment, while not malicious, contains unrelated content. A wide range of MLLMs are evaluated as GUI agents using a simulated dataset, following three working patterns with different levels of perception. Experimental results reveal that even the most powerful models, whether generalist agents or specialist GUI agents, are susceptible to distractions. While recent studies predominantly focus on the helpfulness of agents, our findings first indicate that these agents are prone to environmental distractions. Furthermore, we implement an adversarial environment injection and analyze the approach to improve faithfulness, calling for a collective focus on this important topic. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_02544 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions Ma, Xinbei Wang, Yiting Yao, Yao Yuan, Tongxin Zhang, Aston Zhang, Zhuosheng Zhao, Hai Computation and Language This paper investigates the faithfulness of multimodal large language model (MLLM) agents in a graphical user interface (GUI) environment, aiming to address the research question of whether multimodal GUI agents can be distracted by environmental context. A general scenario is proposed where both the user and the agent are benign, and the environment, while not malicious, contains unrelated content. A wide range of MLLMs are evaluated as GUI agents using a simulated dataset, following three working patterns with different levels of perception. Experimental results reveal that even the most powerful models, whether generalist agents or specialist GUI agents, are susceptible to distractions. While recent studies predominantly focus on the helpfulness of agents, our findings first indicate that these agents are prone to environmental distractions. Furthermore, we implement an adversarial environment injection and analyze the approach to improve faithfulness, calling for a collective focus on this important topic. |
| title | Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2408.02544 |