Cover Image

MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Luera, Reuben A., Rossi, Ryan, Dernoncourt, Franck, Basu, Samyadeep, Kim, Sungchul, Mukherjee, Subhojyoti, Mathur, Puneet, Zhang, Ruiyi, Kil, Jihyung, Lipka, Nedim, Yoon, Seunghyun, Gu, Jiuxiang, Wang, Zichao, Bearfield, Cindy Xiong, Kveton, Branislav
Format:	Preprint
Published:	2025
Subjects:	Human-Computer Interaction Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Reasoning-Based Personalized Generation for Users with Sparse Data
by: Ni, Bo, et al.
Published: (2026)

Survey of User Interface Design and Interaction Techniques in Generative AI Applications
by: Luera, Reuben, et al.
Published: (2024)

Partial Policy Gradients for RL in LLMs
by: Mathur, Puneet, et al.
Published: (2026)

VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-use
by: Zhang, Zhehao, et al.
Published: (2024)

Sparse Personalized Text Generation with Multi-Trajectory Reasoning
by: Ni, Bo, et al.
Published: (2026)

A Survey on LLM-based Conversational User Simulation
by: Ni, Bo, et al.
Published: (2026)

Human-Aligned MLLM Judges for Fine-Grained Image Editing Evaluation: A Benchmark, Framework, and Analysis
by: Liu, Runzhou, et al.
Published: (2026)

ChartLens: Fine-grained Visual Attribution in Charts
by: Suri, Manan, et al.
Published: (2025)

Structured Uncertainty guided Clarification for LLM Agents
by: Suri, Manan, et al.
Published: (2025)

Optimizing Data Delivery: Insights from User Preferences on Visuals, Tables, and Text
by: Luera, Reuben, et al.
Published: (2024)

Agentic Planning with Reasoning for Image Styling via Offline RL
by: Mukherjee, Subhojyoti, et al.
Published: (2026)

A Multi-LLM Debiasing Framework
by: Owens, Deonna M., et al.
Published: (2024)

Follow the Flow: Fine-grained Flowchart Attribution with Neurosymbolic Agents
by: Suri, Manan, et al.
Published: (2025)

MODS: Moderating a Mixture of Document Speakers to Summarize Debatable Queries in Document Collections
by: Balepur, Nishant, et al.
Published: (2025)

Quantitative LLM Judges
by: Sahoo, Aishwarya, et al.
Published: (2025)

Cluster-R1: Large Reasoning Models Are Instruction-following Clustering Agents
by: Qing, Peijun, et al.
Published: (2026)

Efficient and Interpretable Bandit Algorithms
by: Mukherjee, Subhojyoti, et al.
Published: (2023)

Trust but Verify: Introducing DAVinCI -- A Framework for Dual Attribution and Verification in Claim Inference for Language Models
by: Rawte, Vipula, et al.
Published: (2026)

DynaSaur: Large Language Agents Beyond Predefined Actions
by: Nguyen, Dang, et al.
Published: (2024)

StreamGaze: Gaze-Guided Temporal Reasoning and Proactive Understanding in Streaming Videos
by: Lee, Daeun, et al.
Published: (2025)

Document Attribution: Examining Citation Relationships using Large Language Models
by: Rawte, Vipula, et al.
Published: (2025)

Personalization of Large Language Models: A Survey
by: Zhang, Zhehao, et al.
Published: (2024)

MS4UI: A Dataset for Multi-modal Summarization of User Interface Instructional Videos
by: Zang, Yuan, et al.
Published: (2025)

OATS: Opinion Aspect Target Sentiment Quadruple Extraction Dataset for Aspect-Based Sentiment Analysis
by: Chebolu, Siva Uday Sampreeth, et al.
Published: (2023)

ROAST: Review-level Opinion Aspect Sentiment Target Joint Detection for ABSA
by: Chebolu, Siva Uday Sampreeth, et al.
Published: (2024)

Blind to the Human Touch: Overlap Bias in LLM-Based Summary Evaluation
by: Fang, Jiangnan, et al.
Published: (2026)

Test-Time Strategies for More Efficient and Accurate Agentic RAG
by: Zhang, Brian, et al.
Published: (2026)

LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure Profiles
by: Ng, Ho Yin 'Sam', et al.
Published: (2025)

Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models
by: Balasubramanian, Sriram, et al.
Published: (2025)

ChartCitor: Multi-Agent Framework for Fine-Grained Chart Visual Attribution
by: Goswami, Kanika, et al.
Published: (2025)

PlotEdit: Natural Language-Driven Accessible Chart Editing in PDFs via Multimodal LLM Agents
by: Goswami, Kanika, et al.
Published: (2025)

PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback
by: Goswami, Kanika, et al.
Published: (2025)

LLM-as-Judge on a Budget
by: Saha, Aadirupa, et al.
Published: (2026)

LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding
by: Zhang, Yanzhe, et al.
Published: (2023)

KaPQA: Knowledge-Augmented Product Question-Answering
by: Eppalapally, Swetha, et al.
Published: (2024)

From Selection to Generation: A Survey of LLM-based Active Learning
by: Xia, Yu, et al.
Published: (2025)

Charts Are Not Images: On the Challenges of Scientific Chart Editing
by: Li, Shawn, et al.
Published: (2025)

Off-Policy Evaluation from Logged Human Feedback
by: Bhargava, Aniruddha, et al.
Published: (2024)

Multi-Objective Alignment of Large Language Models Through Hypervolume Maximization
by: Mukherjee, Subhojyoti, et al.
Published: (2024)

Anticipatory Planning for Multimodal AI Agents
by: Liang, Yongyuan, et al.
Published: (2026)