Cover Image

UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Yicheng, Anantha, Raviteja, Vashisht, Prabal, Cheng, Jianpeng, Littwin, Etai
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Human-Computer Interaction Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CAMPHOR: Collaborative Agents for Multi-input Planning and High-Order Reasoning On Device
by: Fu, Yicheng, et al.
Published: (2024)

From Text to Self: Users' Perceptions of Potential of AI on Interpersonal Communication and Self
by: Fu, Yue, et al.
Published: (2023)

Augmenting Automation: Intent-Based User Instruction Classification with Machine Learning
by: Basyal, Lochan, et al.
Published: (2024)

MAIC-UI: Making Interactive Courseware with Generative UI
by: Tu, Shangqing, et al.
Published: (2026)

Personality as Relational Infrastructure: User Perceptions of Personality-Trait-Infused LLM Messaging
by: Hofer, Dominik P., et al.
Published: (2026)

MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces
by: Luera, Reuben A., et al.
Published: (2025)

Let Them Down Easy! Contextual Effects of LLM Guardrails on User Perceptions and Preferences
by: Zheng, Mingqian, et al.
Published: (2025)

Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents
by: Qian, Cheng, et al.
Published: (2024)

Morae: Proactively Pausing UI Agents for User Choices
by: Peng, Yi-Hao, et al.
Published: (2025)

User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
by: Wu, Xiaoyuan, et al.
Published: (2025)

Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions
by: Barve, Saharsh, et al.
Published: (2025)

User-Assistant Bias in LLMs
by: Pan, Xu, et al.
Published: (2025)

Vibe Check: Understanding the Effects of LLM-Based Conversational Agents' Personality and Alignment on User Perceptions in Goal-Oriented Tasks
by: Rahman, Hasibur, et al.
Published: (2025)

Do Ethical AI Principles Matter to Users? A Large-Scale Analysis of User Sentiment and Satisfaction
by: Pasch, Stefan, et al.
Published: (2025)

Understand User Opinions of Large Language Models via LLM-Powered In-the-Moment User Experience Interviews
by: Liu, Mengqiao, et al.
Published: (2025)

Generative UI: LLMs are Effective UI Generators
by: Leviathan, Yaniv, et al.
Published: (2026)

HealthGenie: Empowering Users with Healthy Dietary Guidance through Knowledge Graph and Large Language Models
by: Gao, Fan, et al.
Published: (2025)

Comparing How a Chatbot References User Utterances from Previous Chatting Sessions: An Investigation of Users' Privacy Concerns and Perceptions
by: Cox, Samuel Rhys, et al.
Published: (2023)

Creating General User Models from Computer Use
by: Shaikh, Omar, et al.
Published: (2025)

UFO: A UI-Focused Agent for Windows OS Interaction
by: Zhang, Chaoyun, et al.
Published: (2024)

An Analysis of User Behaviors for Objectively Evaluating Spoken Dialogue Systems
by: Inoue, Koji, et al.
Published: (2024)

SparkUI-Parser: Enhancing GUI Perception with Robust Grounding and Parsing
by: Jing, Hongyi, et al.
Published: (2025)

Epistemic Alignment: A Mediating Framework for User-LLM Knowledge Delivery
by: Clark, Nicholas, et al.
Published: (2025)

Speaking the Right Language: The Impact of Expertise Alignment in User-AI Interactions
by: Palta, Shramay, et al.
Published: (2025)

The role of large language models in UI/UX design: A systematic literature review
by: Ahmed, Ammar, et al.
Published: (2025)

Modeling and Optimizing User Preferences in AI Copilots: A Comprehensive Survey and Taxonomy
by: Afzoon, Saleh, et al.
Published: (2025)

Evaluating LLM Adaptation to Sociodemographic Factors: User Profile vs. Dialogue History
by: Zhong, Qishuai, et al.
Published: (2025)

Interactive Recommendation Agent with Active User Commands
by: Tang, Jiakai, et al.
Published: (2025)

EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria
by: Kim, Tae Soo, et al.
Published: (2023)

Generative User-Experience Research for Developing Domain-specific Natural Language Processing Applications
by: Zhukova, Anastasia, et al.
Published: (2023)

CrowdGenUI: Aligning LLM-Based UI Generation with Crowdsourced User Preferences
by: Liu, Yimeng, et al.
Published: (2024)

SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AI
by: Chen, Yunnong, et al.
Published: (2025)

Toward Metaphor-Fluid Conversation Design for Voice User Interfaces
by: Desai, Smit, et al.
Published: (2025)

Human-AI Interaction and User Satisfaction: Empirical Evidence from Online Reviews of AI Products
by: Pasch, Stefan, et al.
Published: (2025)

Planning Ahead with RSA: Efficient Signalling in Dynamic Environments by Projecting User Awareness across Future Timesteps
by: Das, Anwesha, et al.
Published: (2025)

VizTrust: A Visual Analytics Tool for Capturing User Trust Dynamics in Human-AI Communication
by: Wang, Xin, et al.
Published: (2025)

PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue
by: Jeon, Hyunbae, et al.
Published: (2026)

Taxonomy of User Needs and Actions
by: Shelby, Renee, et al.
Published: (2025)

Tell me more: Intent Fulfilment Framework for Enhancing User Experiences in Conversational XAI
by: Wijekoon, Anjana, et al.
Published: (2024)

FocusUI: Efficient UI Grounding via Position-Preserving Visual Token Selection
by: Ouyang, Mingyu, et al.
Published: (2026)