Cover Image

LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yuchong, Orthmann, Bastian, Welle, Michael C., Van Haastregt, Jonne, Kragic, Danica
Format:	Preprint
Published:	2025
Subjects:	Human-Computer Interaction Robotics
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Gesture First, LLM-Assisted Voice Complement: Exploring Multimodal Robot 'Puppeteer' Teleoperation Via Virtual Counterpart in Augmented Reality
by: Zhang, Yuchong, et al.
Published: (2025)

Puppeteer Your Robot: Augmented Reality Leader-Follower Teleoperation
by: van Haastregt, Jonne, et al.
Published: (2024)

Hybrid-Diffusion Models: Combining Open-loop Routines with Visuomotor Diffusion Policies
by: Van Haastregt, Jonne, et al.
Published: (2025)

Reframing Human-Robot Interaction Through Extended Reality: Unlocking Safer, Smarter, and More Empathic Interactions with Virtual Robots and Foundation Models
by: Zhang, Yuchong, et al.
Published: (2025)

Will You Participate? Exploring the Potential of Robotics Competitions on Human-centric Topics
by: Zhang, Yuchong, et al.
Published: (2024)

Vision Beyond Boundaries: An Initial Design Space of Domain-specific Large Vision Models in Human-robot Interaction
by: Zhang, Yuchong, et al.
Published: (2024)

Mind Meets Robots: A Review of EEG-Based Brain-Robot Interaction Systems
by: Zhang, Yuchong, et al.
Published: (2024)

Assisting MoCap-Based Teleoperation of Robot Arm using Augmented Reality Visualisations
by: Zhou, Qiushi, et al.
Published: (2025)

A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
by: Ingelhag, Nils, et al.
Published: (2024)

Augmented Reality User Interface for Command, Control, and Supervision of Large Multi-Agent Teams
by: Regal, Frank, et al.
Published: (2024)

A High-Fidelity Robotic Manipulator Teleoperation Framework for Human-Centered Augmented Reality Evaluation
by: Chhajed, Harsh, et al.
Published: (2026)

IMMERTWIN: A Mixed Reality Framework for Enhanced Robotic Arm Teleoperation
by: Audonnet, Florent P., et al.
Published: (2024)

Early Detection of Human Handover Intentions in Human-Robot Collaboration: Comparing EEG, Gaze, and Hand Motion
by: Khanna, Parag, et al.
Published: (2025)

2024 NASA SUITS Report: LLM-Driven Immersive Augmented Reality User Interface for Robotics and Space Exploration
by: Zhuang, Kathy, et al.
Published: (2025)

PuppetAI: A Customizable Platform for Designing Tactile-Rich Affective Robot Interaction
by: Li, Jiaye, et al.
Published: (2026)

Indicating Robot Vision Capabilities with Augmented Reality
by: Wang, Hong, et al.
Published: (2025)

Towards Human-Robot Teaming through Augmented Reality and Gaze-Based Attention Control
by: Shleibik, Yousra, et al.
Published: (2024)

Robotic versus Human Teleoperation for Remote Ultrasound
by: Black, David, et al.
Published: (2025)

Understanding and Mitigating Network Latency Effect on Teleoperated-Robot with Extended Reality
by: Zhang, Ziliang, et al.
Published: (2025)

EVE: Enabling Anyone to Train Robots using Augmented Reality
by: Wang, Jun, et al.
Published: (2024)

Toward a Predictive eXtended Reality Teleoperation System with Duo-Virtual Spaces
by: Zhang, Ziliang, et al.
Published: (2024)

AZRA: Extending the Affective Capabilities of Zoomorphic Robots using Augmented Reality
by: Macdonald, Shaun, et al.
Published: (2025)

Explainable OOHRI: Communicating Robot Capabilities and Limitations as Augmented Reality Affordances
by: Wang, Lauren W., et al.
Published: (2026)

Investigating the Influence of Spatial Ability in Augmented Reality-assisted Robot Programming
by: Leins, Nicolas, et al.
Published: (2026)

Confidence-based Intent Prediction for Teleoperation in Bimanual Robotic Suturing
by: Hu, Zhaoyang Jacopo, et al.
Published: (2025)

Walk along: An Experiment on Controlling the Mobile Robot 'Spot' with Voice and Gestures
by: Zhang, Renchi, et al.
Published: (2024)

Advancing User-Voice Interaction: Exploring Emotion-Aware Voice Assistants Through a Role-Swapping Approach
by: Ma, Yong, et al.
Published: (2025)

Optimizing Robot Programming: Mixed Reality Gripper Control
by: Rettinger, Maximilian, et al.
Published: (2025)

Stability and Transparency in Mixed Reality Bilateral Human Teleoperation
by: Black, David Gregory, et al.
Published: (2024)

The Conversation is the Command: Interacting with Real-World Autonomous Robot Through Natural Language
by: Nwankwo, Linus, et al.
Published: (2024)

Going Down the Abstraction Stream with Augmented Reality and Tangible Robots: the Case of Vector Instruction
by: Volodin, Sergei, et al.
Published: (2025)

GhostObjects: Instructing Robots by Manipulating Spatially Aligned Virtual Twins in Augmented Reality
by: Wang, Lauren W., et al.
Published: (2025)

Self-Centering 3-DoF Feet Controller for Hands-Free Locomotion Control in Telepresence and Virtual Reality
by: Memmesheimer, Raphael, et al.
Published: (2024)

TeleopLab: Accessible and Intuitive Teleoperation of a Robotic Manipulator for Remote Labs
by: Chen, Ziling, et al.
Published: (2025)

HapCompass: A Rotational Haptic Device for Contact-Rich Robotic Teleoperation
by: Tan, Xiangshan, et al.
Published: (2026)

Should Teleoperation Be like Driving in a Car? Comparison of Teleoperation HMIs
by: Wolf, Maria-Magdalena, et al.
Published: (2024)

Feasibility of Augmented Reality-Guided Robotic Ultrasound with Cone-Beam CT Integration for Spine Procedures
by: Song, Tianyu, et al.
Published: (2026)

Motion Control of Interactive Robotic Arms Based on Mixed Reality Development
by: Chen, Hanxiao
Published: (2024)

The Ballad of the Bots: Sonification Using Cognitive Metaphor to Support Immersed Teleoperation of Robot Teams
by: Simmons, Joe, et al.
Published: (2024)

Effects of Shared Control on Cognitive Load and Trust in Teleoperated Trajectory Tracking
by: Pan, Jiahe, et al.
Published: (2024)