SEAR: A Multimodal Dataset for Analyzing AR-LLM-Driven Social Engineering Behaviors

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yu, Tianlong, Ye, Chenghang, Yang, Zheyu, Zhou, Ziyi, Tang, Cui, Tao, Zui, Zhang, Jun, Wang, Kailong, Zhou, Liting, Yang, Yang, Bi, Ting
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866912404156186624
author	Yu, Tianlong Ye, Chenghang Yang, Zheyu Zhou, Ziyi Tang, Cui Tao, Zui Zhang, Jun Wang, Kailong Zhou, Liting Yang, Yang Bi, Ting
author_facet	Yu, Tianlong Ye, Chenghang Yang, Zheyu Zhou, Ziyi Tang, Cui Tao, Zui Zhang, Jun Wang, Kailong Zhou, Liting Yang, Yang Bi, Ting
contents	The SEAR Dataset is a novel multimodal resource designed to study the emerging threat of social engineering (SE) attacks orchestrated through augmented reality (AR) and multimodal large language models (LLMs). This dataset captures 180 annotated conversations across 60 participants in simulated adversarial scenarios, including meetings, classes and networking events. It comprises synchronized AR-captured visual/audio cues (e.g., facial expressions, vocal tones), environmental context, and curated social media profiles, alongside subjective metrics such as trust ratings and susceptibility assessments. Key findings reveal SEAR's alarming efficacy in eliciting compliance (e.g., 93.3% phishing link clicks, 85% call acceptance) and hijacking trust (76.7% post-interaction trust surge). The dataset supports research in detecting AR-driven SE attacks, designing defensive frameworks, and understanding multimodal adversarial manipulation. Rigorous ethical safeguards, including anonymization and IRB compliance, ensure responsible use. The SEAR dataset is available at https://github.com/INSLabCN/SEAR-Dataset.
format	Preprint
id	arxiv_https___arxiv_org_abs_2505_24458
institution	arXiv
publishDate	2025
record_format	arxiv
spellingShingle	SEAR: A Multimodal Dataset for Analyzing AR-LLM-Driven Social Engineering Behaviors Yu, Tianlong Ye, Chenghang Yang, Zheyu Zhou, Ziyi Tang, Cui Tao, Zui Zhang, Jun Wang, Kailong Zhou, Liting Yang, Yang Bi, Ting Artificial Intelligence The SEAR Dataset is a novel multimodal resource designed to study the emerging threat of social engineering (SE) attacks orchestrated through augmented reality (AR) and multimodal large language models (LLMs). This dataset captures 180 annotated conversations across 60 participants in simulated adversarial scenarios, including meetings, classes and networking events. It comprises synchronized AR-captured visual/audio cues (e.g., facial expressions, vocal tones), environmental context, and curated social media profiles, alongside subjective metrics such as trust ratings and susceptibility assessments. Key findings reveal SEAR's alarming efficacy in eliciting compliance (e.g., 93.3% phishing link clicks, 85% call acceptance) and hijacking trust (76.7% post-interaction trust surge). The dataset supports research in detecting AR-driven SE attacks, designing defensive frameworks, and understanding multimodal adversarial manipulation. Rigorous ethical safeguards, including anonymization and IRB compliance, ensure responsible use. The SEAR dataset is available at https://github.com/INSLabCN/SEAR-Dataset.
title	SEAR: A Multimodal Dataset for Analyzing AR-LLM-Driven Social Engineering Behaviors
topic	Artificial Intelligence
url	https://arxiv.org/abs/2505.24458