Cover Image

Gesture First, LLM-Assisted Voice Complement: Exploring Multimodal Robot 'Puppeteer' Teleoperation Via Virtual Counterpart in Augmented Reality

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Yuchong, Orthmann, Bastian, Ji, Shichen, Welle, Michael, Van Haastregt, Jonne, Kragic, Danica
Format:	Preprint
Published:	2025
Subjects:	Human-Computer Interaction Robotics
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation
by: Zhang, Yuchong, et al.
Published: (2025)

Puppeteer Your Robot: Augmented Reality Leader-Follower Teleoperation
by: van Haastregt, Jonne, et al.
Published: (2024)

Hybrid-Diffusion Models: Combining Open-loop Routines with Visuomotor Diffusion Policies
by: Van Haastregt, Jonne, et al.
Published: (2025)

A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models
by: Ingelhag, Nils, et al.
Published: (2024)

Reframing Human-Robot Interaction Through Extended Reality: Unlocking Safer, Smarter, and More Empathic Interactions with Virtual Robots and Foundation Models
by: Zhang, Yuchong, et al.
Published: (2025)

Will You Participate? Exploring the Potential of Robotics Competitions on Human-centric Topics
by: Zhang, Yuchong, et al.
Published: (2024)

Advancing User-Voice Interaction: Exploring Emotion-Aware Voice Assistants Through a Role-Swapping Approach
by: Ma, Yong, et al.
Published: (2025)

Low-Cost Teleoperation with Haptic Feedback through Vision-based Tactile Sensors for Rigid and Soft Object Manipulation
by: Lippi, Martina, et al.
Published: (2024)

S$^2$-Diffusion: Generalizing from Instance-level to Category-level Skills in Robot Manipulation
by: Yang, Quantao, et al.
Published: (2025)

Preference Aligned Visuomotor Diffusion Policies for Deformable Object Manipulation
by: Moletta, Marco, et al.
Published: (2026)

Learning Dexterous In-Hand Manipulation with Multifingered Hands via Visuomotor Diffusion
by: Koczy, Piotr, et al.
Published: (2025)

Vision Beyond Boundaries: An Initial Design Space of Domain-specific Large Vision Models in Human-robot Interaction
by: Zhang, Yuchong, et al.
Published: (2024)

AdaFold: Adapting Folding Trajectories of Cloths via Feedback-loop Manipulation
by: Longhini, Alberta, et al.
Published: (2024)

FLAME: A Federated Learning Benchmark for Robotic Manipulation
by: Betran, Santiago Bou, et al.
Published: (2025)

Standardization of Cloth Objects and its Relevance in Robotic Manipulation
by: Garcia-Camacho, Irene, et al.
Published: (2024)

The 1st InterAI Workshop: Interactive AI for Human-centered Robotics
by: Zhang, Yuchong, et al.
Published: (2024)

Mind Meets Robots: A Review of EEG-Based Brain-Robot Interaction Systems
by: Zhang, Yuchong, et al.
Published: (2024)

Real-Time Operator Takeover for Visuomotor Diffusion Policy Training
by: Moletta, Marco, et al.
Published: (2025)

On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning
by: Lips, Thomas, et al.
Published: (2026)

Personalizing Emotion-aware Conversational Agents? Exploring User Traits-driven Conversational Strategies for Enhanced Interaction
by: Zhang, Yuchong, et al.
Published: (2025)

Assisting MoCap-Based Teleoperation of Robot Arm using Augmented Reality Visualisations
by: Zhou, Qiushi, et al.
Published: (2025)

Raising Body Ownership in End-to-End Visuomotor Policy Learning via Robot-Centric Pooling
by: Zhuang, Zheyu, et al.
Published: (2024)

Visual Action Planning with Multiple Heterogeneous Agents
by: Lippi, Martina, et al.
Published: (2024)

Observer Design for Augmented Reality-based Teleoperation of Soft Robots
by: García-Samartín, Jorge Francisco, et al.
Published: (2026)

Augmented Reality-Enhanced Robot Teleoperation for Collecting User Demonstrations
by: Gong, Shiqi, et al.
Published: (2025)

User-customizable Shared Control for Robot Teleoperation via Virtual Reality
by: Luo, Rui, et al.
Published: (2024)

Gesture Evaluation in Virtual Reality
by: Werner, Axel Wiebe, et al.
Published: (2025)

Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
by: Jaquier, Noémie, et al.
Published: (2023)

Geometry of Uncertainty: Learning Metric Spaces for Multimodal State Estimation in RL
by: Reichlin, Alfredo, et al.
Published: (2026)

BEAVR: Bimanual, multi-Embodiment, Accessible, Virtual Reality Teleoperation System for Robots
by: Posadas-Nava, Alejandro, et al.
Published: (2025)

EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamics
by: Longhini, Alberta, et al.
Published: (2022)

Evaluating Gesture Recognition in Virtual Reality
by: Sabbella, Sandeep Reddy, et al.
Published: (2024)

Real-Time Iteration Scheme for Diffusion Policy
by: Duan, Yufei, et al.
Published: (2025)

Walking on the Fiber: A Simple Geometric Approximation for Bayesian Neural Networks
by: Reichlin, Alfredo, et al.
Published: (2025)

A High-Fidelity Robotic Manipulator Teleoperation Framework for Human-Centered Augmented Reality Evaluation
by: Chhajed, Harsh, et al.
Published: (2026)

GesPrompt: Leveraging Co-Speech Gestures to Augment LLM-Based Interaction in Virtual Reality
by: Hu, Xiyun, et al.
Published: (2025)

Self-supervised 6-DoF Robot Grasping by Demonstration via Augmented Reality Teleoperation System
by: Dengxiong, Xiwen, et al.
Published: (2024)

Unfolding the Literature: A Review of Robotic Cloth Manipulation
by: Longhini, Alberta, et al.
Published: (2024)

Accessible Gesture-Driven Augmented Reality Interaction System
by: Wang, Yikan
Published: (2025)

Ring-width-index of five west african tree species, Aguima Catchment, Benin
by: Orthmann, Bettina
Published: (2014)