A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908301239779328 |
|---|---|
| author | Mohammed, Abdul Mannan Mohammad, Azhar Ali Ortiz, Jason A. Neumann, Carsten Bochenek, Grace Reiners, Dirk Cruz-Neira, Carolina |
| author_facet | Mohammed, Abdul Mannan Mohammad, Azhar Ali Ortiz, Jason A. Neumann, Carsten Bochenek, Grace Reiners, Dirk Cruz-Neira, Carolina |
| contents | Recent developments in Artificial Intelligence (AI) and Machine Learning (ML) are creating new opportunities for Human-Autonomy Teaming (HAT) in tasks, missions, and continuous coordinated activities. A major challenge is enabling humans to maintain awareness and control over autonomous assets, while also building trust and supporting shared contextual understanding. To address this, we present a real-time Human Digital Twin (HDT) architecture that integrates Large Language Models (LLMs) for knowledge reporting, answering, and recommendation, embodied in a visual interface.
The system applies a metacognitive approach to enable personalized, context-aware responses aligned with the human teammate's expectations. The HDT acts as a visually and behaviorally realistic team member, integrated throughout the mission lifecycle, from training to deployment to after-action review. Our architecture includes speech recognition, context processing, AI-driven dialogue, emotion modeling, lip-syncing, and multimodal feedback. We describe the system design, performance metrics, and future development directions for more adaptive and realistic HAT systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_03147 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations Mohammed, Abdul Mannan Mohammad, Azhar Ali Ortiz, Jason A. Neumann, Carsten Bochenek, Grace Reiners, Dirk Cruz-Neira, Carolina Human-Computer Interaction Artificial Intelligence Recent developments in Artificial Intelligence (AI) and Machine Learning (ML) are creating new opportunities for Human-Autonomy Teaming (HAT) in tasks, missions, and continuous coordinated activities. A major challenge is enabling humans to maintain awareness and control over autonomous assets, while also building trust and supporting shared contextual understanding. To address this, we present a real-time Human Digital Twin (HDT) architecture that integrates Large Language Models (LLMs) for knowledge reporting, answering, and recommendation, embodied in a visual interface. The system applies a metacognitive approach to enable personalized, context-aware responses aligned with the human teammate's expectations. The HDT acts as a visually and behaviorally realistic team member, integrated throughout the mission lifecycle, from training to deployment to after-action review. Our architecture includes speech recognition, context processing, AI-driven dialogue, emotion modeling, lip-syncing, and multimodal feedback. We describe the system design, performance metrics, and future development directions for more adaptive and realistic HAT systems. |
| title | A Human Digital Twin Architecture for Knowledge-based Interactions and Context-Aware Conversations |
| topic | Human-Computer Interaction Artificial Intelligence |
| url | https://arxiv.org/abs/2504.03147 |