"Can you be my mum?": Manipulating Social Robots in the Large Language Models Era
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910777195102208 |
|---|---|
| author | Abbo, Giulio Antonio Desideri, Gloria Belpaeme, Tony Spitale, Micol |
| author_facet | Abbo, Giulio Antonio Desideri, Gloria Belpaeme, Tony Spitale, Micol |
| contents | Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_04633 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | "Can you be my mum?": Manipulating Social Robots in the Large Language Models Era Abbo, Giulio Antonio Desideri, Gloria Belpaeme, Tony Spitale, Micol Human-Computer Interaction Computers and Society Robotics Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions. |
| title | "Can you be my mum?": Manipulating Social Robots in the Large Language Models Era |
| topic | Human-Computer Interaction Computers and Society Robotics |
| url | https://arxiv.org/abs/2501.04633 |