Cover Image

UITron-Speech: Towards Automated GUI Agents Based on Speech Instructions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Han, Wenkang, Zeng, Zhixiong, Huang, Jing, Jiang, Shu, Zheng, Liming, Yang, Longrong, Qiu, Haibo, Yao, Chang, Chen, Jingyuan, Ma, Lin
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ScaleTrack: Scaling and back-tracking Automated GUI Agents
by: Huang, Jing, et al.
Published: (2025)

UItron: Foundational GUI Agent with Advanced Perception and Planning
by: Zeng, Zhixiong, et al.
Published: (2025)

OmniActor: A Generalist GUI and Embodied Agent for 2D&3D Worlds
by: Yang, Longrong, et al.
Published: (2025)

TreeCUA: Efficiently Scaling GUI Automation with Tree-Structured Verifiable Evolution
by: Jiang, Deyang, et al.
Published: (2026)

MobileDreamer: Generative Sketch World Model for GUI Agent
by: Cao, Yilin, et al.
Published: (2026)

Agentic Reward Modeling: Verifying GUI Agent via Online Proactive Interaction
by: Cui, Chaoqun, et al.
Published: (2026)

DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarios
by: Zhong, Yufeng, et al.
Published: (2025)

OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Models
by: Zhong, Yufeng, et al.
Published: (2026)

Towards Trustworthy GUI Agents: A Survey
by: Shi, Yucheng, et al.
Published: (2025)

VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning
by: Zhao, Xuanle, et al.
Published: (2025)

Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCR
by: Zhong, Yufeng, et al.
Published: (2025)

DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning
by: Zhang, Chi, et al.
Published: (2025)

VideoGUI: A Benchmark for GUI Automation from Instructional Videos
by: Lin, Kevin Qinghong, et al.
Published: (2024)

Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation
by: Chen, Lei, et al.
Published: (2025)

Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
by: Ma, Xiaoxiao, et al.
Published: (2025)

DKT2: Revisiting Applicable and Comprehensive Knowledge Tracing in Large-Scale Data
by: Zhou, Yiyun, et al.
Published: (2025)

A$_3$B$_2$: Adaptive Asymmetric Adapter for Alleviating Branch Bias in Vision-Language Image Classification with Few-Shot Learning
by: Zhou, Yiyun, et al.
Published: (2026)

Towards Unsupervised Speech Recognition at the Syllable-Level
by: Wang, Liming, et al.
Published: (2025)

Mobile-Agent-v3: Fundamental Agents for GUI Automation
by: Ye, Jiabo, et al.
Published: (2025)

STAGE: Stable and Generalizable GRPO for Autoregressive Image Generation
by: Ma, Xiaoxiao, et al.
Published: (2025)

S2S-Arena: Evaluating Paralinguistic Instruction Following in Speech-to-Speech Models
by: Jiang, Feng, et al.
Published: (2025)

Show and Polish: Reference-Guided Identity Preservation in Face Video Restoration
by: Han, Wenkang, et al.
Published: (2025)

Towards Unsupervised Speech Recognition Without Pronunciation Models
by: Ni, Junrui, et al.
Published: (2024)

AUTO-Explorer: Automated Data Collection for GUI Agent
by: Guo, Xiangwu, et al.
Published: (2025)

Using GUI Agent for Electronic Design Automation
by: Li, Chunyi, et al.
Published: (2025)

UI-TARS: Pioneering Automated GUI Interaction with Native Agents
by: Qin, Yujia, et al.
Published: (2025)

SpeechCaps: Advancing Instruction-Based Universal Speech Models with Multi-Talker Speaking Style Captioning
by: Huang, Chien-yu, et al.
Published: (2024)

Metis-SPECS: Decoupling Multimodal Learning via Self-distilled Preference-based Cold Start
by: Chen, Kun, et al.
Published: (2025)

MOSS-Speech: Towards True Speech-to-Speech Models Without Text Guidance
by: Zhao, Xingjian, et al.
Published: (2025)

AdaZoom-GUI: Adaptive Zoom-based GUI Grounding with Instruction Refinement
by: Pei, Siqi, et al.
Published: (2026)

SpeechAgent: An End-to-End Mobile Infrastructure for Speech Impairment Assistance
by: Lou, Haowei, et al.
Published: (2025)

Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speech
by: Huang, Chien-yu, et al.
Published: (2023)

DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data
by: Lu, Ke-Han, et al.
Published: (2024)

Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning
by: Zhang, Chi, et al.
Published: (2025)

Towards Robust Dysarthric Speech Recognition: LLM-Agent Post-ASR Correction Beyond WER
by: Zheng, Xiuwen, et al.
Published: (2026)

CoCo-Agent: A Comprehensive Cognitive MLLM Agent for Smartphone GUI Automation
by: Ma, Xinbei, et al.
Published: (2024)

Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Models
by: Lu, Ke-Han, et al.
Published: (2025)

FlexSpeech: Towards Stable, Controllable and Expressive Text-to-Speech
by: Ma, Linhan, et al.
Published: (2025)

LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospects
by: Liu, Guangyi, et al.
Published: (2025)

Toward Fair Speech Technologies: A Comprehensive Survey of Bias and Fairness in Speech AI
by: Lin, Yi-Cheng, et al.
Published: (2026)