Imagen de Portada

Beyond Screenshots: Evaluating VLMs' Understanding of UI Animations

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liang, Chen, Jiang, Xirui, Deng, Naihao, Adar, Eytan, Guo, Anhong
Formato:	Preprint
Publicado:	2026
Materias:	Human-Computer Interaction Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

QuizRank: Picking Images by Quizzing VLMs
por: Ji, Tenghao, et al.
Publicado: (2025)

Feminist Interaction Techniques: Deterring Non-Consensual Screenshots with Interaction Techniques
por: Qiwei, Li, et al.
Publicado: (2024)

Exploring Bridges Between Algorithmic and AI-generated Art
por: Wu, Jiaqi, et al.
Publicado: (2024)

Assessing Affective Objectives for Communicative Visualizations
por: Lee-Robbins, Elsie, et al.
Publicado: (2026)

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
por: You, Keen, et al.
Publicado: (2024)

WorldScribe: Towards Context-Aware Live Visual Descriptions
por: Chang, Ruei-Che, et al.
Publicado: (2024)

Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters
por: Zhang, Xingjian, et al.
Publicado: (2025)

Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms
por: Ashkinaze, Joshua, et al.
Publicado: (2024)

MAIC-UI: Making Interactive Courseware with Generative UI
por: Tu, Shangqing, et al.
Publicado: (2026)

UI-Evol: Automatic Knowledge Evolving for Computer Use Agents
por: Zhang, Ziyun, et al.
Publicado: (2025)

ReDemon UI: Reactive Synthesis by Demonstration for Web UI
por: Lee, Jay, et al.
Publicado: (2025)

EditScribe: Non-Visual Image Editing with Natural Language Verification Loops
por: Chang, Ruei-Che, et al.
Publicado: (2024)

One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations
por: Lee, Yoonjoo, et al.
Publicado: (2024)

HandProxy: Expanding the Affordances of Speech Interfaces in Immersive Environments with a Virtual Proxy Hand
por: Liang, Chen, et al.
Publicado: (2025)

ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
por: Hsiao, Yu-Chung, et al.
Publicado: (2022)

ProgramAlly: Creating Custom Visual Access Programs via Multi-Modal End-User Programming
por: Herskovitz, Jaylin, et al.
Publicado: (2024)

The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyond
por: Chen, Xiang 'Anthony', et al.
Publicado: (2025)

StateScribe: Towards Accessible Change Awareness Across Real-World Revisits
por: Chang, Ruei-Che, et al.
Publicado: (2026)

Imagine a dragon made of seaweed: How images enhance learning in Wikipedia
por: Silva, Anita, et al.
Publicado: (2024)

Generative UI: LLMs are Effective UI Generators
por: Leviathan, Yaniv, et al.
Publicado: (2026)

MUIAnno: An Expert-Annotated Dataset and Evaluation Benchmark for Mobile UI Understanding
por: Parvez, Athar, et al.
Publicado: (2026)

Animating Language Practice: Engagement with Stylized Conversational Agents in Japanese Learning
por: Rackauckas, Zackary, et al.
Publicado: (2025)

UFO: A UI-Focused Agent for Windows OS Interaction
por: Zhang, Chaoyun, et al.
Publicado: (2024)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
por: Vo, Truong, et al.
Publicado: (2025)

AURORA: Navigating UI Tarpits via Automated Neural Screen Understanding
por: Khan, Safwat Ali, et al.
Publicado: (2024)

RWKV-UI: UI Understanding with Enhanced Perception and Reasoning
por: Yang, Jiaxi, et al.
Publicado: (2025)

Morae: Proactively Pausing UI Agents for User Choices
por: Peng, Yi-Hao, et al.
Publicado: (2025)

The role of large language models in UI/UX design: A systematic literature review
por: Ahmed, Ammar, et al.
Publicado: (2025)

Macaron-A2UI: A Model for Generative UI in Personal Agents
por: Kong, Fancy, et al.
Publicado: (2026)

FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
por: Ouyang, Mingyu, et al.
Publicado: (2026)

SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
por: Chen, Yunnong, et al.
Publicado: (2025)

From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts
por: Zhang, Zhuohao Jerry, et al.
Publicado: (2024)

Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts
por: Kim, Taewook, et al.
Publicado: (2024)

What Social Media Use Do People Regret? An Analysis of 34K Smartphone Screenshots with Multimodal LLM
por: Guo, Longjie, et al.
Publicado: (2024)

Human-AI Narrative Synthesis to Foster Shared Understanding in Civic Decision-Making
por: Overney, Cassandra, et al.
Publicado: (2025)

UI-TARS: Pioneering Automated GUI Interaction with Native Agents
por: Qin, Yujia, et al.
Publicado: (2025)

UI-UG: A Unified MLLM for UI Understanding and Generation
por: Yang, Hao, et al.
Publicado: (2025)

AutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
por: Zhang, Tianyi, et al.
Publicado: (2024)

MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling
por: Feng, Sidong, et al.
Publicado: (2024)

GestureGPT: Toward Zero-Shot Free-Form Hand Gesture Understanding with Large Language Model Agents
por: Zeng, Xin, et al.
Publicado: (2023)