Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.13112 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917107234504704 |
|---|---|
| author | Wei, Wenya Yang, Sipeng Zhou, Qixian Liu, Ruochen Zhang, Xuelei Yuan, Yifu Jiang, Yan Luo, Yongle Wang, Hailong Wang, Tianzhou Jin, Peipei Liu, Wangtong Zhao, Zhou Jin, Xiaogang Liu, Elvis S. |
| author_facet | Wei, Wenya Yang, Sipeng Zhou, Qixian Liu, Ruochen Zhang, Xuelei Yuan, Yifu Jiang, Yan Luo, Yongle Wang, Hailong Wang, Tianzhou Jin, Peipei Liu, Wangtong Zhao, Zhou Jin, Xiaogang Liu, Elvis S. |
| contents | In cooperative video games, traditional AI companions are deployed to assist players, who control them using hotkeys or command wheels to issue predefined commands such as ``attack'', ``defend'', or ``retreat''. Despite their simplicity, these methods, which lack target specificity, limit players' ability to give complex tactical instructions and hinder immersive gameplay experiences. To address this problem, we propose the FPS AI Companion who Understands Language (F.A.C.U.L.), the first real-time AI system that enables players to communicate and collaborate with AI companions using natural language. By integrating natural language processing with a confidence-based framework, F.A.C.U.L. efficiently decomposes complex commands and interprets player intent. It also employs a dynamic entity retrieval method for environmental awareness, aligning human intentions with decision-making. Unlike traditional rule-based systems, our method supports real-time language interactions, enabling players to issue complex commands such as ``clear the second floor'', ``take cover behind that tree'', or ``retreat to the river''. The system provides real-time behavioral responses and vocal feedback, ensuring seamless tactical collaboration. Using the popular FPS game \textit{Arena Breakout: Infinite} as a case study, we present comparisons demonstrating the efficacy of our approach and discuss the advantages and limitations of AI companions based on real-world user feedback. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_13112 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | F.A.C.U.L.: Language-Based Interaction with AI Companions in Gaming Wei, Wenya Yang, Sipeng Zhou, Qixian Liu, Ruochen Zhang, Xuelei Yuan, Yifu Jiang, Yan Luo, Yongle Wang, Hailong Wang, Tianzhou Jin, Peipei Liu, Wangtong Zhao, Zhou Jin, Xiaogang Liu, Elvis S. Human-Computer Interaction In cooperative video games, traditional AI companions are deployed to assist players, who control them using hotkeys or command wheels to issue predefined commands such as ``attack'', ``defend'', or ``retreat''. Despite their simplicity, these methods, which lack target specificity, limit players' ability to give complex tactical instructions and hinder immersive gameplay experiences. To address this problem, we propose the FPS AI Companion who Understands Language (F.A.C.U.L.), the first real-time AI system that enables players to communicate and collaborate with AI companions using natural language. By integrating natural language processing with a confidence-based framework, F.A.C.U.L. efficiently decomposes complex commands and interprets player intent. It also employs a dynamic entity retrieval method for environmental awareness, aligning human intentions with decision-making. Unlike traditional rule-based systems, our method supports real-time language interactions, enabling players to issue complex commands such as ``clear the second floor'', ``take cover behind that tree'', or ``retreat to the river''. The system provides real-time behavioral responses and vocal feedback, ensuring seamless tactical collaboration. Using the popular FPS game \textit{Arena Breakout: Infinite} as a case study, we present comparisons demonstrating the efficacy of our approach and discuss the advantages and limitations of AI companions based on real-world user feedback. |
| title | F.A.C.U.L.: Language-Based Interaction with AI Companions in Gaming |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2511.13112 |