Cover Image

Guiding Multimodal Large Language Models with Blind and Low Vision People Visual Questions for Proactive Visual Interpretations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Penuela, Ricardo Gonzalez, Arias-Russi, Felipe, Capriles, Victor
Format:	Preprint
Published:	2025
Subjects:	Computer Vision and Pattern Recognition Artificial Intelligence Human-Computer Interaction I.2.m; H.5.2
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Towards Understanding the Use of MLLM-Enabled Applications for Visual Interpretation by Blind and Low Vision People
by: Penuela, Ricardo E. Gonzalez, et al.
Published: (2025)

How Multimodal Large Language Models Support Access to Visual Information: A Diary Study With Blind and Low Vision People
by: Penuela, Ricardo E. Gonzalez, et al.
Published: (2026)

Autonomous Mapping and Navigation using Fiducial Markers and Pan-Tilt Camera for Assisting Indoor Mobility of Blind and Visually Impaired People
by: Adapa, Dharmateja, et al.
Published: (2023)

Visual Bias in Simulated Users: The Impact of Luminance and Contrast on Reinforcement Learning-based Interaction
by: Selder, Hannah, et al.
Published: (2026)

OOPrompt: Reifying Intents into Structured Artifacts for Modular and Iterative Prompting
by: Xu, Tengyou, et al.
Published: (2026)

Investigating Use Cases of AI-Powered Scene Description Applications for Blind and Low Vision People
by: Gonzalez, Ricardo, et al.
Published: (2024)

TapNav: Adaptive Spatiotactile Screen Readers for Tactually Guided Touchscreen Interactions for Blind and Low Vision People
by: Gonzalez, Ricardo, et al.
Published: (2025)

ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing
by: Arawjo, Ian, et al.
Published: (2023)

LACE: Controlled Image Prompting and Iterative Refinement with GenAI for Professional Visual Art Creators
by: Huang, Yenkai, et al.
Published: (2025)

AdaptLIL: A Gaze-Adaptive Visualization for Ontology Mapping
by: Chow, Nicholas, et al.
Published: (2024)

Vision-Based Hand Gesture Customization from a Single Demonstration
by: Shahi, Soroush, et al.
Published: (2024)

Distorted Perspectives of LLM-Simulated Preferences: Can AI Mislead Design?
by: Kuric, Eduard, et al.
Published: (2026)

SocialLM: Social Signal Processing of Patient-Provider Communication using LLMs and Contextual Aggregation
by: Bedmutha, Manas Satish, et al.
Published: (2025)

Inclusive Kitchen Design for Older Adults: Generative AI Visualizations to Support Mild Cognitive Impairment
by: Bilau, Ibrahim, et al.
Published: (2026)

Semantic Reality: Interactive Context-Aware Visualization of Inter-Object Relationships in Augmented Reality
by: Liu, Xiaoan, et al.
Published: (2026)

GazePrompt: Enhancing Low Vision People's Reading Experience with Gaze-Aware Augmentations
by: Wang, Ru, et al.
Published: (2024)

"The Data Says Otherwise"-Towards Automated Fact-checking and Communication of Data Claims
by: Fu, Yu, et al.
Published: (2024)

What Makes a Model Breathe? Understanding Reinforcement Learning Reward Function Design in Biomechanical User Simulation
by: Selder, Hannah, et al.
Published: (2025)

Mind & Motion: Opportunities and Applications of Integrating Biomechanics and Cognitive Models in HCI
by: Fleig, Arthur, et al.
Published: (2025)

A Retrospective on Ultrasound Mid-Air Haptics in HCI
by: Fleig, Arthur
Published: (2025)

Demystifying Reward Design in Reinforcement Learning for Upper Extremity Interaction: Practical Guidelines for Biomechanical Simulations in HCI
by: Selder, Hannah, et al.
Published: (2025)

Human Agency, Causality, and the Human Computer Interface in High-Stakes Artificial Intelligence
by: Hattab, Georges
Published: (2026)

Exploring Mobile Touch Interaction with Large Language Models
by: Zindulka, Tim, et al.
Published: (2025)

Texterial: A Text-as-Material Interaction Paradigm for LLM-Mediated Writing
by: Shen, Jocelyn, et al.
Published: (2026)

Real-Time World Crafting: Generating Structured Game Behaviors from Natural Language with Large Language Models
by: Drake, Austin, et al.
Published: (2025)

GamerAstra: Supporting 2D Non-Twitch Video Games for Blind and Low-Vision Players through a Multi-Agent Framework
by: Qiu, Tianrun, et al.
Published: (2025)

ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators
by: Li, Franklin Mingzhe, et al.
Published: (2026)

CommentScope: A Comment-Embedded Assisted Reading System for a Long Text
by: Chen, Shuai, et al.
Published: (2025)

StudyAlign: A Software System for Conducting Web-Based User Studies with Functional Interactive Prototypes
by: Lehmann, Florian, et al.
Published: (2025)

GenFaceUI: Meta-Design of Generative Personalized Facial Expression Interfaces for Intelligent Agents
by: Ge, Yate, et al.
Published: (2026)

Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs through Conversations, Toolbars, and Prompts
by: Lehmann, Florian, et al.
Published: (2024)

FlyMeThrough: Human-AI Collaborative 3D Indoor Mapping with Commodity Drones
by: Su, Xia, et al.
Published: (2025)

Can LLMs and humans be friends? Uncovering factors affecting human-AI intimacy formation
by: Hong, Yeseon, et al.
Published: (2025)

"I Felt Bad After We Ignored Her": Understanding How Interface-Driven Social Prominence Shapes Group Discussions with GenAI
by: Johnson, Janet G., et al.
Published: (2026)

LLMs Enable Context-Aware Augmented Reality in Surgical Navigation
by: Javaheri, Hamraz, et al.
Published: (2024)

Interactive Formal Specification for Mathematical Problems of Engineers
by: Neuper, Walther
Published: (2024)

IRL Dittos: Embodied Multimodal AI Agent Interactions in Open Spaces
by: Lee, Seonghee, et al.
Published: (2025)

TS-Insight: Visualizing Thompson Sampling for Verification and XAI
by: Vares, Parsa, et al.
Published: (2025)

Does Embodiment Matter to Biomechanics and Function? A Comparative Analysis of Head-Mounted and Hand-Held Assistive Devices for Individuals with Blindness and Low Vision
by: Seth, Gaurav, et al.
Published: (2025)

MAESTRO: Adapting GUIs and Guiding Navigation with User Preferences in Conversational Agents with GUIs
by: Lee, Sangwook, et al.
Published: (2026)