Copertina

Prompt2Task: Automating UI Tasks on Smartphones from Textual Prompts

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Huang, Tian, Yu, Chun, Shi, Weinan, Peng, Zijian, Yang, David, Sun, Weiqi, Shi, Yuanchun
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Human-Computer Interaction
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

AngleSizer: Enhancing Spatial Scale Perception for the Visually Impaired with an Interactive Smartphone Assistant
di: Jing, Xiaoqing, et al.
Pubblicazione: (2024)

LlamaTouch: A Faithful and Scalable Testbed for Mobile UI Task Automation
di: Zhang, Li, et al.
Pubblicazione: (2024)

TaskSense: Cognitive Chain Modeling and Difficulty Estimation for GUI Tasks
di: Yin, Yiwen, et al.
Pubblicazione: (2025)

VisionTasker: Mobile Task Automation Using Vision Based UI Understanding and LLM Task Planning
di: Song, Yunpeng, et al.
Pubblicazione: (2023)

TextOnly: A Unified Function Portal for Text-Related Functions on Smartphones
di: Tu, Minghao, et al.
Pubblicazione: (2025)

CAAP: Context-Aware Action Planning Prompting to Solve Computer Tasks with Front-End UI Only
di: Cho, Junhee, et al.
Pubblicazione: (2024)

A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples
di: Pan, Lihang, et al.
Pubblicazione: (2024)

SonarWatch: Field sensing technique for smartwatches based on ultrasound and motion
di: Shi, Yingtian, et al.
Pubblicazione: (2024)

Dynamic Prompt Middleware: Contextual Prompt Refinement Controls for Comprehension Tasks
di: Drosos, Ian, et al.
Pubblicazione: (2024)

Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task
di: Zhu, Mengxiao, et al.
Pubblicazione: (2024)

From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote Task Understanding
di: Zhang, Nandi, et al.
Pubblicazione: (2025)

Bridging the gap between natural user expression with complex automation programming in smart homes
di: Shi, Yingtian, et al.
Pubblicazione: (2024)

Say Your Reason: Extract Contextual Rules In Situ for Context-aware Service Recommendation
di: Li, Yuxuan, et al.
Pubblicazione: (2024)

DuetUI: A Bidirectional Context Loop for Human-Agent Co-Generation of Task-Oriented Interfaces
di: Xu, Yuan, et al.
Pubblicazione: (2025)

SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
di: Chen, Yunnong, et al.
Pubblicazione: (2025)

PACT: Proactive Asking for Continual Task Assistance in Human-Robot Collaboration
di: He, Chengbo, et al.
Pubblicazione: (2026)

Understanding Prompt Programming Tasks and Questions
di: Liang, Jenny T., et al.
Pubblicazione: (2025)

CasualGaze: Towards Modeling and Recognizing Casual Gaze Behavior for Efficient Gaze-based Object Selection
di: Shi, Yingtian, et al.
Pubblicazione: (2024)

GazeSummary: Exploring Gaze as an Implicit Prompt for Personalization in Text-based LLM Tasks
di: Ding, Jiexin, et al.
Pubblicazione: (2026)

Agent-Initiated Interaction in Phone UI Automation
di: Kahlon, Noam, et al.
Pubblicazione: (2025)

PoseAugment: Generative Human Pose Data Augmentation with Physical Plausibility for IMU-based Motion Capture
di: Li, Zhuojun, et al.
Pubblicazione: (2024)

Can AI Prompt Humans? Multimodal Agents Prompt Players' Game Actions and Show Consequences to Raise Sustainability Awareness
di: Zhang, Qinshi, et al.
Pubblicazione: (2024)

MapAgent: Trajectory-Constructed Memory-Augmented Planning for Mobile Task Automation
di: Kong, Yi, et al.
Pubblicazione: (2025)

MLLM-Based UI2Code Automation Guided by UI Layout Information
di: Wu, Fan, et al.
Pubblicazione: (2025)

Advancing GUI for Generative AI: Charting the Design Space of Human-AI Interactions through Task Creativity and Complexity
di: Ding, Zijian
Pubblicazione: (2024)

Towards Intent-based User Interfaces: Charting the Design Space of Intent-AI Interactions Across Task Types
di: Ding, Zijian
Pubblicazione: (2024)

Struggle First, Prompt Later: How Task Complexity Shapes Learning with GenAI-Assisted Pretesting
di: Akgun, Mahir, et al.
Pubblicazione: (2025)

When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design Task
di: Han, Yuanning, et al.
Pubblicazione: (2025)

Automating UI Optimization through Multi-Agentic Reasoning
di: Li, Zhipeng, et al.
Pubblicazione: (2026)

Automated UI Interface Generation via Diffusion Models: Enhancing Personalization and Efficiency
di: Duan, Yifei, et al.
Pubblicazione: (2025)

Macaron-A2UI: A Model for Generative UI in Personal Agents
di: Kong, Fancy, et al.
Pubblicazione: (2026)

Leveraging Large Language Models for Generating Mobile Sensing Strategies in Human Behavior Modeling
di: Gao, Nan, et al.
Pubblicazione: (2023)

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
di: Suzgun, Mirac, et al.
Pubblicazione: (2024)

Adapting AI to the Moment: Understanding the Dynamics of Parent-AI Collaboration Modes in Real-Time Conversations with Children
di: Mei, Yu, et al.
Pubblicazione: (2026)

Time2Stop: Adaptive and Explainable Human-AI Loop for Smartphone Overuse Intervention
di: Orzikulova, Adiba, et al.
Pubblicazione: (2024)

From Prompt Engineering to Prompt Craft
di: Lindley, Joseph, et al.
Pubblicazione: (2024)

Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings
di: Elizabeth, Michelle, et al.
Pubblicazione: (2024)

CoGen: Creation of Reusable UI Components in Figma via Textual Commands
di: Kanapathipillai, Ishani, et al.
Pubblicazione: (2026)

Exploring and Analyzing the Effect of Avatar's Visual Style on Anxiety of English as Second Language (ESL) Speakers
di: Liu, Tianqi, et al.
Pubblicazione: (2023)

SituFont: A Just-in-Time Adaptive Intervention System for Enhancing Mobile Readability in Situational Visual Impairments
di: Chen, Jingruo, et al.
Pubblicazione: (2024)