'Can you be my mum?' : manipulating social robots in the large language models era
- Author
- Giulio Antonio Abbo (UGent) , Gloria Desideri, Tony Belpaeme (UGent) and Micol Spitale
- Organization
- Project
- Abstract
- Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions.
- Keywords
- red-teaming, manipulation, large language model, LLM, ethical principles, social norms, social robots
Downloads
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 1.19 MB
-
DS918 acc.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 558.58 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01JY6D7S3VR4547ZF43Y16VGAH
- MLA
- Abbo, Giulio Antonio, et al. “’Can You Be My Mum?’ : Manipulating Social Robots in the Large Language Models Era.” 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI, IEEE, 2025, pp. 1181–85, doi:10.1109/hri61500.2025.10973919.
- APA
- Abbo, G. A., Desideri, G., Belpaeme, T., & Spitale, M. (2025). ’Can you be my mum?’ : manipulating social robots in the large language models era. 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI, 1181–1185. https://doi.org/10.1109/hri61500.2025.10973919
- Chicago author-date
- Abbo, Giulio Antonio, Gloria Desideri, Tony Belpaeme, and Micol Spitale. 2025. “’Can You Be My Mum?’ : Manipulating Social Robots in the Large Language Models Era.” In 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI, 1181–85. IEEE. https://doi.org/10.1109/hri61500.2025.10973919.
- Chicago author-date (all authors)
- Abbo, Giulio Antonio, Gloria Desideri, Tony Belpaeme, and Micol Spitale. 2025. “’Can You Be My Mum?’ : Manipulating Social Robots in the Large Language Models Era.” In 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI, 1181–1185. IEEE. doi:10.1109/hri61500.2025.10973919.
- Vancouver
- 1.Abbo GA, Desideri G, Belpaeme T, Spitale M. ’Can you be my mum?’ : manipulating social robots in the large language models era. In: 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI. IEEE; 2025. p. 1181–5.
- IEEE
- [1]G. A. Abbo, G. Desideri, T. Belpaeme, and M. Spitale, “’Can you be my mum?’ : manipulating social robots in the large language models era,” in 2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI, Melbourne, Australia, 2025, pp. 1181–1185.
@inproceedings{01JY6D7S3VR4547ZF43Y16VGAH,
abstract = {{Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions.}},
author = {{Abbo, Giulio Antonio and Desideri, Gloria and Belpaeme, Tony and Spitale, Micol}},
booktitle = {{2025 20TH ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION, HRI}},
isbn = {{9798350378948}},
issn = {{2167-2121}},
keywords = {{red-teaming,manipulation,large language model,LLM,ethical principles,social norms,social robots}},
language = {{eng}},
location = {{Melbourne, Australia}},
pages = {{1181--1185}},
publisher = {{IEEE}},
title = {{'Can you be my mum?' : manipulating social robots in the large language models era}},
url = {{http://doi.org/10.1109/hri61500.2025.10973919}},
year = {{2025}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: