Cover Image

See-Control: A Multimodal Agent Framework for Smartphone Interaction with a Robotic Arm

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhao, Haoyu, Ding, Weizhong, Yang, Yuhao, Tian, Zheng, Yang, Linyi, Shao, Kun, Wang, Jun
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computer Vision and Pattern Recognition Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Arm Robot: AR-Enhanced Embodied Control and Visualization for Intuitive Robot Arm Manipulation
by: Pei, Siyou, et al.
Published: (2024)

Integrated Control of Robotic Arm through EMG and Speech: Decision-Driven Multimodal Data Fusion
by: Mohd, Tauheed Khan, et al.
Published: (2024)

SmartPoser: Arm Pose Estimation with a Smartphone and Smartwatch Using UWB and IMU Data
by: DeVrio, Nathan, et al.
Published: (2025)

See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles
by: Wu, Zongru, et al.
Published: (2025)

A Dialogue-Based Human-Robot Interaction Protocol for Wheelchair and Robotic Arm Integrated Control
by: Liu, Guangping, et al.
Published: (2026)

User Experience Estimation in Human-Robot Interaction Via Multi-Instance Learning of Multimodal Social Signals
by: Miyoshi, Ryo, et al.
Published: (2025)

Inferring Operator Emotions from a Motion-Controlled Robotic Arm
by: Qi, Xinyu, et al.
Published: (2025)

See Behind Walls in Real-time Using Aerial Drones and Augmented Reality
by: Yang, Sikai, et al.
Published: (2024)

DeepSORT-Driven Visual Tracking Approach for Gesture Recognition in Interactive Systems
by: Zhang, Tong, et al.
Published: (2025)

STAR: Smartphone-analogous Typing in Augmented Reality
by: Kim, Taejun, et al.
Published: (2025)

Algorithmic Ways of Seeing: Using Object Detection to Facilitate Art Exploration
by: Meyer, Louie Søs, et al.
Published: (2024)

Seeing, Hearing, and Knowing Together: Multimodal Strategies in Deepfake Videos Detection
by: Chen, Chen, et al.
Published: (2026)

UIPro: Unleashing Superior Interaction Capability For GUI Agents
by: Li, Hongxin, et al.
Published: (2025)

LLM Agent-Based Simulation of Student Activities and Mental Health Using Smartphone Sensing Data
by: Sommuang, Wayupuk, et al.
Published: (2025)

Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models
by: Xie, Jingyi, et al.
Published: (2025)

InteractOut: Leveraging Interaction Proxies as Input Manipulation Strategies for Reducing Smartphone Overuse
by: Lu, Tao, et al.
Published: (2024)

AngleSizer: Enhancing Spatial Scale Perception for the Visually Impaired with an Interactive Smartphone Assistant
by: Jing, Xiaoqing, et al.
Published: (2024)

Human-Robot Collaboration for the Remote Control of Mobile Humanoid Robots with Torso-Arm Coordination
by: Boguslavskii, Nikita, et al.
Published: (2025)

SpatialViz-Bench: A Cognitively-Grounded Benchmark for Diagnosing Spatial Visualization in MLLMs
by: Wang, Siting, et al.
Published: (2025)

IMMERTWIN: A Mixed Reality Framework for Enhanced Robotic Arm Teleoperation
by: Audonnet, Florent P., et al.
Published: (2024)

Designing for Rich Collocated Social Interactions in the Age of Smartphones
by: Genç, Hüseyin Uğur
Published: (2024)

See Through Their Minds: Learning Transferable Neural Representation from Cross-Subject fMRI
by: Liu, Yulong, et al.
Published: (2024)

iLearnRobot: An Interactive Learning-Based Multi-Modal Robot with Continuous Improvement
by: Wang, Kohou, et al.
Published: (2025)

Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction
by: Mascaro, Esteve Valls, et al.
Published: (2024)

Designing for Patient Voice in Interactive Health
by: Sun, Yuhao
Published: (2026)

Impact of Target and Tool Visualization on Depth Perception and Usability in Optical See-Through AR
by: Yang, Yue, et al.
Published: (2025)

Prompt2Task: Automating UI Tasks on Smartphones from Textual Prompts
by: Huang, Tian, et al.
Published: (2024)

Exploring a Design Framework for Children's Agency through Participatory Design
by: Yang, Boyin, et al.
Published: (2026)

AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning
by: Becerra, Alvaro, et al.
Published: (2025)

AnyUser: Translating Sketched User Intent into Domestic Robots
by: Yang, Songyuan, et al.
Published: (2026)

Relative Attention-based One-Class Adversarial Autoencoder for Continuous Authentication of Smartphone Users
by: Hu, Mingming, et al.
Published: (2022)

ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation
by: Xia, Ding, et al.
Published: (2025)

MotionTrace: IMU-based Field of View Prediction for Smartphone AR Interactions
by: Islam, Rahul, et al.
Published: (2024)

Plug-and-Play Clarifier: A Zero-Shot Multimodal Framework for Egocentric Intent Disambiguation
by: Yang, Sicheng, et al.
Published: (2025)

ObjectFinder: An Open-Vocabulary Assistive System for Interactive Object Search by Blind People
by: Liu, Ruiping, et al.
Published: (2024)

MindCine: Multimodal EEG-to-Video Reconstruction with Large-Scale Pretrained Models
by: Zhou, Tian-Yi, et al.
Published: (2026)

Multimodal Interaction and Intention Communication for Industrial Robots
by: Schreiter, Tim, et al.
Published: (2025)

Evaluating Sensitivity Parameters in Smartphone-Based Gaze Estimation: A Comparative Study of Appearance-Based and Infrared Eye Trackers
by: Gunawardena, Nishan, et al.
Published: (2025)

InterFeedback: Unveiling Interactive Intelligence of Large Multimodal Models via Human Feedback
by: Zhao, Henry Hengyuan, et al.
Published: (2025)

Give me scissors: Collision-Free Dual-Arm Surgical Assistive Robot for Instrument Delivery
by: Luo, Xuejin, et al.
Published: (2026)