Cover Image

FAM-HRI: Foundation-Model Assisted Multi-Modal Human-Robot Interaction Combining Gaze and Speech

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lai, Yuzhi, Yuan, Shenghai, Li, Peizheng, Zhang, Boya, Kiefer, Benjamin, Deng, Tianchen, Zell, Andreas
Format:	Preprint
Published:	2025
Subjects:	Human-Computer Interaction Robotics
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

SemanticScanpath: Combining Gaze and Speech for Situated Human-Robot Interaction Using LLMs
by: Menendez, Elisabeth, et al.
Published: (2025)

Gaze4HRI: Zero-shot Benchmarking Gaze Estimation Neural-Networks for Human-Robot Interaction
by: Sezer, Berk, et al.
Published: (2026)

Sticky-Glance: Robust Intent Recognition for Human Robot Collaboration via Single-Glance
by: Lai, Yuzhi, et al.
Published: (2026)

Human-Robot Interaction Conversational User Enjoyment Scale (HRI CUES)
by: Irfan, Bahar, et al.
Published: (2024)

Architectural HRI: Towards a Robotic Paradigm Shift in Human-Building Interaction
by: Nguyen, Alex Binh Vinh Duc
Published: (2026)

Teaching Introductory HRI: UChicago Course "Human-Robot Interaction: Research and Practice"
by: Sebo, Sarah
Published: (2024)

Talking to Robots: A Practical Examination of Speech Foundation Models for HRI Applications
by: Rosin, Theresa Pekarek, et al.
Published: (2025)

Toward Family-Robot Interactions: A Family-Centered Framework in HRI
by: Cagiltay, Bengisu, et al.
Published: (2024)

Gaze-based intention estimation: principles, methodologies, and applications in HRI
by: Belardinelli, Anna
Published: (2023)

Gazing at Failure: Investigating Human Gaze in Response to Robot Failure in Collaborative Tasks
by: Tabatabaei, Ramtin, et al.
Published: (2025)

Gaze-supported Large Language Model Framework for Bi-directional Human-Robot Interaction
by: Rüppel, Jens V., et al.
Published: (2025)

Plant-Inspired Robot Design Metaphors for Ambient HRI
by: Antony, Victor Nikhil, et al.
Published: (2026)

GEAR: Gaze-Enabled Human-Robot Collaborative Assembly
by: Shahid, Asad Ali, et al.
Published: (2025)

Gaze-Based Intention Recognition for Human-Robot Collaboration
by: Belcamino, Valerio, et al.
Published: (2024)

Paper index: Designing an introductory HRI course (workshop at HRI 2024)
by: Admoni, Henny, et al.
Published: (2024)

Speech-Gesture Mapping and Engagement Evaluation in Human Robot Interaction
by: Ghosh, Bishal, et al.
Published: (2018)

RaycastGrasp: Eye-Gaze Interaction with Wearable Devices for Robotic Manipulation
by: Lin, Zitiantao, et al.
Published: (2025)

A Checklist for Deploying Robots in Public: Articulating Tacit Knowledge in the HRI Community
by: Liang, Claire, et al.
Published: (2026)

Using vs. Purchasing Industrial Robots: Adding an Organizational Perspective to Industrial HRI
by: Hostettler, Damian
Published: (2024)

ERR@HRI 2.0 Challenge: Multimodal Detection of Errors and Failures in Human-Robot Conversations
by: Cao, Shiye, et al.
Published: (2025)

LaMI: Large Language Models for Multi-Modal Human-Robot Interaction
by: Wang, Chao, et al.
Published: (2024)

HRI Challenges Influencing Low Usage of Robotic Systems in Disaster Response and Rescue Operations
by: Hoque, Shahinul, et al.
Published: (2024)

Sense4HRI: A ROS 2 HRI Framework for Physiological Sensor Integration and Synchronized Logging
by: Scheibl, Manuel, et al.
Published: (2026)

Vid2Real HRI: Align video-based HRI study designs with real-world settings
by: Hauser, Elliott, et al.
Published: (2024)

Modeling Subjective Urban Perception with Human Gaze
by: Che, Lin, et al.
Published: (2026)

HAGI++: Head-Assisted Gaze Imputation and Generation
by: Jiao, Chuhan, et al.
Published: (2025)

Transferability of HRI Research: Potential and Challenges
by: Johal, Wafa
Published: (2024)

HRI in Indian Education: Challenges Opportunities
by: Mishra, Chinmaya, et al.
Published: (2024)

A Multi-Modal Interaction Framework for Efficient Human-Robot Collaborative Shelf Picking
by: Pathak, Abhinav, et al.
Published: (2025)

Looking for the Human in HRI Teaching: User-Centered Course Design for Tech-Savvy Students
by: Doernbach, Tobias
Published: (2024)

Design Principles & Issues for Gaze and Pinch Interaction
by: Pfeuffer, Ken
Published: (2024)

From Pets to Robots: MojiKit as a Data-Informed Toolkit for Affective HRI Design
by: He, Liwen, et al.
Published: (2026)

Early Detection of Human Handover Intentions in Human-Robot Collaboration: Comparing EEG, Gaze, and Hand Motion
by: Khanna, Parag, et al.
Published: (2025)

Factored Reasoning with Inner Speech and Persistent Memory for Evidence-Grounded Human-Robot Interaction
by: Belcamino, Valerio, et al.
Published: (2026)

Towards Human-Robot Teaming through Augmented Reality and Gaze-Based Attention Control
by: Shleibik, Yousra, et al.
Published: (2024)

IntenBot: Flexible and Imprecise Multimodal Input for LLMs to Understand User Intentions for Casual and Human-Like HRI
by: Liu, Yen-Ting, et al.
Published: (2026)

Human-Like Gaze Behavior in Social Robots: A Deep Learning Approach Integrating Human and Non-Human Stimuli
by: Vahedi, Faezeh, et al.
Published: (2026)

Reframing Human-Robot Interaction Through Extended Reality: Unlocking Safer, Smarter, and More Empathic Interactions with Virtual Robots and Foundation Models
by: Zhang, Yuchong, et al.
Published: (2025)

Child Speech Recognition in Human-Robot Interaction: Problem Solved?
by: Janssens, Ruben, et al.
Published: (2024)

The People's Gaze: Co-Designing and Refining Gaze Gestures with General Users and Gaze Interaction Experts
by: Lei, Yaxiong, et al.
Published: (2026)