Image de couverture de livre

Beyond Screenshots: Evaluating VLMs' Understanding of UI Animations

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Liang, Chen, Jiang, Xirui, Deng, Naihao, Adar, Eytan, Guo, Anhong
Format:	Preprint
Publié:	2026
Sujets:	Human-Computer Interaction Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

QuizRank: Picking Images by Quizzing VLMs
par: Ji, Tenghao, et autres
Publié: (2025)

Feminist Interaction Techniques: Deterring Non-Consensual Screenshots with Interaction Techniques
par: Qiwei, Li, et autres
Publié: (2024)

Exploring Bridges Between Algorithmic and AI-generated Art
par: Wu, Jiaqi, et autres
Publié: (2024)

Assessing Affective Objectives for Communicative Visualizations
par: Lee-Robbins, Elsie, et autres
Publié: (2026)

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
par: You, Keen, et autres
Publié: (2024)

WorldScribe: Towards Context-Aware Live Visual Descriptions
par: Chang, Ruei-Che, et autres
Publié: (2024)

Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
par: Zhang, Xingjian, et autres
Publié: (2025)

Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms
par: Ashkinaze, Joshua, et autres
Publié: (2024)

MAIC-UI: Making Interactive Courseware with Generative UI
par: Tu, Shangqing, et autres
Publié: (2026)

UI-Evol: Automatic Knowledge Evolving for Computer Use Agents
par: Zhang, Ziyun, et autres
Publié: (2025)

ReDemon UI: Reactive Synthesis by Demonstration for Web UI
par: Lee, Jay, et autres
Publié: (2025)

EditScribe: Non-Visual Image Editing with Natural Language Verification Loops
par: Chang, Ruei-Che, et autres
Publié: (2024)

One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations
par: Lee, Yoonjoo, et autres
Publié: (2024)

HandProxy: Expanding the Affordances of Speech Interfaces in Immersive Environments with a Virtual Proxy Hand
par: Liang, Chen, et autres
Publié: (2025)

ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
par: Hsiao, Yu-Chung, et autres
Publié: (2022)

ProgramAlly: Creating Custom Visual Access Programs via Multi-Modal End-User Programming
par: Herskovitz, Jaylin, et autres
Publié: (2024)

The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyond
par: Chen, Xiang 'Anthony', et autres
Publié: (2025)

StateScribe: Towards Accessible Change Awareness Across Real-World Revisits
par: Chang, Ruei-Che, et autres
Publié: (2026)

Imagine a dragon made of seaweed: How images enhance learning in Wikipedia
par: Silva, Anita, et autres
Publié: (2024)

Generative UI: LLMs are Effective UI Generators
par: Leviathan, Yaniv, et autres
Publié: (2026)

MUIAnno: An Expert-Annotated Dataset and Evaluation Benchmark for Mobile UI Understanding
par: Parvez, Athar, et autres
Publié: (2026)

Animating Language Practice: Engagement with Stylized Conversational Agents in Japanese Learning
par: Rackauckas, Zackary, et autres
Publié: (2025)

UFO: A UI-Focused Agent for Windows OS Interaction
par: Zhang, Chaoyun, et autres
Publié: (2024)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
par: Vo, Truong, et autres
Publié: (2025)

AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding
par: Khan, Safwat Ali, et autres
Publié: (2024)

RWKV-UI: UI Understanding with Enhanced Perception and Reasoning
par: Yang, Jiaxi, et autres
Publié: (2025)

Morae: Proactively Pausing UI Agents for User Choices
par: Peng, Yi-Hao, et autres
Publié: (2025)

The role of large language models in UI/UX design: A systematic literature review
par: Ahmed, Ammar, et autres
Publié: (2025)

Macaron-A2UI: A Model for Generative UI in Personal Agents
par: Kong, Fancy, et autres
Publié: (2026)

FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
par: Ouyang, Mingyu, et autres
Publié: (2026)

SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
par: Chen, Yunnong, et autres
Publié: (2025)

From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts
par: Zhang, Zhuohao Jerry, et autres
Publié: (2024)

Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts
par: Kim, Taewook, et autres
Publié: (2024)

What Social Media Use Do People Regret? An Analysis of 34K Smartphone Screenshots with Multimodal LLM
par: Guo, Longjie, et autres
Publié: (2024)

Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making
par: Overney, Cassandra, et autres
Publié: (2025)

UI-TARS: Pioneering Automated GUI Interaction with Native Agents
par: Qin, Yujia, et autres
Publié: (2025)

UI-UG: A Unified MLLM for UI Understanding and Generation
par: Yang, Hao, et autres
Publié: (2025)

AutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
par: Zhang, Tianyi, et autres
Publié: (2024)

MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling
par: Feng, Sidong, et autres
Publié: (2024)

GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents
par: Zeng, Xin, et autres
Publié: (2023)