Image de couverture de livre

OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Jiahao Nick, Xu, Yan, Grossman, Tovi, Santosa, Stephanie, Li, Michelle
Format:	Preprint
Publié:	2024
Sujets:	Human-Computer Interaction Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering
par: Li, Jiahao Nick, et autres
Publié: (2024)

LOOM: Personalized Learning Informed by Daily LLM Conversations Toward Long-Term Mastery via a Dynamic Learner Memory Graph
par: Cui, Justin, et autres
Publié: (2025)

CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs
par: Kazemitabaar, Majeed, et autres
Publié: (2024)

StoryEnsemble: Enabling Dynamic Exploration & Iteration in the Design Process with AI and Forward-Backward Propagation
par: Suh, Sangho, et autres
Publié: (2025)

Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning
par: Kazemitabaar, Majeed, et autres
Publié: (2024)

KnowledgeTrail: Generative Timeline for Exploration and Sensemaking of Historical Events and Knowledge Formation
par: Suh, Sangho, et autres
Publié: (2025)

Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition
par: Kazemitabaar, Majeed, et autres
Publié: (2024)

SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration
par: Brade, Stephen, et autres
Publié: (2023)

Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support
par: Pu, Kevin, et autres
Publié: (2025)

Desk2Desk: Optimization-based Mixed Reality Workspace Integration for Remote Side-by-side Collaboration
par: Sidenmark, Ludwig, et autres
Publié: (2024)

Towards Real-World Validity in Generative AI Benchmarks: Understanding and Designing Domain-Centered Evaluations for Journalism Practitioners
par: Li, Charlotte, et autres
Publié: (2025)

Characterizing Human Actions in the Digital Platform by Temporal Context
par: Matsui, Akira, et autres
Publié: (2022)

OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions
par: Luo, Cheng, et autres
Publié: (2025)

Script2Screen: Supporting Dialogue Scriptwriting with Interactive Audiovisual Generation
par: Wang, Zhecheng, et autres
Publié: (2025)

ACE, Action and Control via Explanations: A Proposal for LLMs to Provide Human-Centered Explainability for Multimodal AI Assistants
par: Watkins, Elizabeth Anne, et autres
Publié: (2025)

MaRginalia: Enabling In-person Lecture Capturing and Note-taking Through Mixed Reality
par: Qiu, Leping, et autres
Publié: (2025)

Adaptique: Multi-objective and Context-aware Online Adaptation of Selection Techniques in Virtual Reality
par: Lai, Chao-Jung, et autres
Publié: (2025)

ImageInThat: Manipulating Images to Convey User Instructions to Robots
par: Mahadevan, Karthik, et autres
Publié: (2025)

When LLMs Can't Help: Real-World Evaluation of LLMs in Nutrition
par: Li, Karen Jia-Hui, et autres
Publié: (2025)

IdeaSynth: Iterative Research Idea Development Through Evolving and Composing Idea Facets with Literature-Grounded Feedback
par: Pu, Kevin, et autres
Publié: (2024)

TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback
par: Mokhtar, Sassan, et autres
Publié: (2026)

You Only Look at Screens: Multimodal Chain-of-Action Agents
par: Zhang, Zhuosheng, et autres
Publié: (2023)

Mind the Value-Action Gap: Do LLMs Act in Alignment with Their Values?
par: Shen, Hua, et autres
Publié: (2025)

An HCAI Methodological Framework (HCAI-MF): Putting It Into Action to Enable Human-Centered AI
par: Xu, Wei, et autres
Publié: (2023)

ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions
par: Yang, Bufang, et autres
Publié: (2025)

Addressing Climate Action Misperceptions with Generative AI
par: Remshard, Miriam, et autres
Publié: (2026)

OmniGUI: Benchmarking GUI Agents in Omni-Modal Smartphone Environments
par: Henry, Felix, et autres
Publié: (2026)

Personagram: Bridging Personas and Product Design for Creative Ideation with Multimodal LLMs
par: Kim, Taewook, et autres
Publié: (2026)

Think-Then-React: Towards Unconstrained Human Action-to-Reaction Generation
par: Tan, Wenhui, et autres
Publié: (2025)

See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMs
par: Li, Zhimin, et autres
Publié: (2025)

OpenOmni: A Collaborative Open Source Tool for Building Future-Ready Multimodal Conversational Agents
par: Sun, Qiang, et autres
Publié: (2024)

See2Refine: Vision-Language Feedback Improves LLM-Based eHMI Action Designers
par: Xia, Ding, et autres
Publié: (2026)

MLLM as a UI Judge: Benchmarking Multimodal LLMs for Predicting Human Perception of User Interfaces
par: Luera, Reuben A., et autres
Publié: (2025)

ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models
par: Reza, Mohi, et autres
Publié: (2023)

Simulating Human Behavior with the Psychological-mechanism Agent: Integrating Feeling, Thought, and Action
par: Dong, Qing, et autres
Publié: (2025)

Context-Value-Action Architecture for Value-Driven Large Language Model Agents
par: Zhang, TianZe, et autres
Publié: (2026)

MeetMap: Real-Time Collaborative Dialogue Mapping with LLMs in Online Meetings
par: Chen, Xinyue, et autres
Publié: (2025)

Harms in Repurposing Real-World Sensory Cues for Mixed Reality: A Causal Perspective
par: Tao, Yujie, et autres
Publié: (2024)

Developing Situational Awareness for Joint Action with Autonomous Vehicles
par: Kaufman, Robert, et autres
Publié: (2024)

Automating eHMI Action Design with LLMs for Automated Vehicle Communication
par: Xia, Ding, et autres
Publié: (2025)