Cover Image

UI-UG: A Unified MLLM for UI Understanding and Generation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Yang, Hao, Qiu, Weijie, Zhang, Ru, Fang, Zhou, Mao, Ruichao, Lin, Xiaoyu, Huang, Maji, Huang, Zhaosong, Guo, Teng, Liu, Shuoyang, Rao, Hai
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

MLLM-Based UI2Code Automation Guided by UI Layout Information
by: Wu, Fan, et al.
Published: (2025)

RWKV-UI: UI Understanding with Enhanced Perception and Reasoning
by: Yang, Jiaxi, et al.
Published: (2025)

ReDemon UI: Reactive Synthesis by Demonstration for Web UI
by: Lee, Jay, et al.
Published: (2025)

Macaron-A2UI: A Model for Generative UI in Personal Agents
by: Kong, Fancy, et al.
Published: (2026)

Beyond Screenshots: Evaluating VLMs' Understanding of UI Animations
by: Liang, Chen, et al.
Published: (2026)

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
by: You, Keen, et al.
Published: (2024)

Privacy Starts with UI: Privacy Patterns and Designer Perspectives in UI/UX Practice
by: Maloku, Anxhela, et al.
Published: (2026)

UI Remix: Supporting UI Design Through Interactive Example Retrieval and Remixing
by: Wang, Junling, et al.
Published: (2026)

CrowdGenUI: Aligning LLM-Based UI Generation with Crowdsourced User Preferences
by: Liu, Yimeng, et al.
Published: (2024)

GhostUI: Unveiling Hidden Interactions in Mobile UI
by: Kweon, Minkyu, et al.
Published: (2026)

MAIC-UI: Making Interactive Courseware with Generative UI
by: Tu, Shangqing, et al.
Published: (2026)

The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyond
by: Chen, Xiang 'Anthony', et al.
Published: (2025)

SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
by: Chen, Yunnong, et al.
Published: (2025)

User-Centric Design of UI for Mobile Banking Apps: Improving UI and Features for Better Customer Experience
by: Chitrakar, Luniva, et al.
Published: (2026)

MUD: Towards a Large-Scale and Noise-Filtered UI Dataset for Modern Style UI Modeling
by: Feng, Sidong, et al.
Published: (2024)

What is the focus of XAI in UI design? Prioritizing UI design principles for enhancing XAI user experience
by: Lei, Dian, et al.
Published: (2024)

MUIAnno: An Expert-Annotated Dataset and Evaluation Benchmark for Mobile UI Understanding
by: Parvez, Athar, et al.
Published: (2026)

Generative UI: LLMs are Effective UI Generators
by: Leviathan, Yaniv, et al.
Published: (2026)

MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces
by: Luera, Reuben A., et al.
Published: (2025)

Preference-Guided Multi-Objective UI Adaptation
by: Song, Yao, et al.
Published: (2025)

Agent-Initiated Interaction in Phone UI Automation
by: Kahlon, Noam, et al.
Published: (2025)

Aria-UI: Visual Grounding for GUI Instructions
by: Yang, Yuhao, et al.
Published: (2024)

Prompt2Task: Automating UI Tasks on Smartphones from Textual Prompts
by: Huang, Tian, et al.
Published: (2024)

ConsentDiff at Scale: Longitudinal Audits of Web Privacy Policy Changes and UI Frictions
by: Guo, Haoze
Published: (2025)

ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent
by: Zhao, Yuheng, et al.
Published: (2025)

Misty: UI Prototyping Through Interactive Conceptual Blending
by: Lu, Yuwen, et al.
Published: (2024)

Automating UI Optimization through Multi-Agentic Reasoning
by: Li, Zhipeng, et al.
Published: (2026)

SparkUI-Parser: Enhancing GUI Perception with Robust Grounding and Parsing
by: Jing, Hongyi, et al.
Published: (2025)

AutoGameUI: Constructing High-Fidelity GameUI via Multimodal Correspondence Matching
by: Tang, Zhongliang, et al.
Published: (2024)

FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
by: Ouyang, Mingyu, et al.
Published: (2026)

Generating Automatic Feedback on UI Mockups with Large Language Models
by: Duan, Peitong, et al.
Published: (2024)

Affordances of Sketched Notations for Multimodal UI Design and Development Tools
by: Ross, Sam H., et al.
Published: (2025)

DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing
by: Vaithilingam, Priyan, et al.
Published: (2024)

VisionTasker: Mobile Task Automation Using Vision Based UI Understanding and LLM Task Planning
by: Song, Yunpeng, et al.
Published: (2023)

FEAD: Figma-Enhanced App Design Framework for Improving UI/UX in Educational App Development
by: Huang, Tianyi
Published: (2024)

SPRITE: From Static Mockups to Engine-Ready Game UI
by: Bai, Yunshu, et al.
Published: (2026)

From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts
by: Zhang, Zhuohao Jerry, et al.
Published: (2024)

UI-Evol: Automatic Knowledge Evolving for Computer Use Agents
by: Zhang, Ziyun, et al.
Published: (2025)

Open WebUI: An Open, Extensible, and Usable Interface for AI Interaction
by: Baek, Jaeryang, et al.
Published: (2025)

Dancing With Chains: Ideating Under Constraints With UIDEC in UI/UX Design
by: Shokrizadeh, Atefeh, et al.
Published: (2025)