Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909149376282624 |
|---|---|
| author | Li, Fan Feng, Shanshan Yan, Yuqi Lee, Ching-Hung Ong, Yew Soon |
| author_facet | Li, Fan Feng, Shanshan Yan, Yuqi Lee, Ching-Hung Ong, Yew Soon |
| contents | Advancements in technology, pilot shortages, and cost pressures are driving a trend towards single-pilot and even remote operations in aviation. Considering the extensive workload and huge risks associated with single-pilot operations, the development of a Virtual Co-Pilot (V-CoP) is expected to be a potential way to ensure aviation safety. This study proposes a V-CoP concept and explores how humans and virtual assistants can effectively collaborate. A preliminary case study is conducted to explore a critical role of V-CoP, namely automated quick procedures searching, using the multimodal large language model (LLM). The LLM-enabled V-CoP integrates the pilot instruction and real-time cockpit instrumental data to prompt applicable aviation manuals and operation procedures. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis and effective retrieval of procedure information. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis (90.5%) and effective retrieval of procedure information (86.5%). The proposed V-CoP is expected to provide a foundation for future virtual intelligent assistant development, improve the performance of single pilots, and reduce the risk of human errors in aviation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_16645 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations Li, Fan Feng, Shanshan Yan, Yuqi Lee, Ching-Hung Ong, Yew Soon Human-Computer Interaction Advancements in technology, pilot shortages, and cost pressures are driving a trend towards single-pilot and even remote operations in aviation. Considering the extensive workload and huge risks associated with single-pilot operations, the development of a Virtual Co-Pilot (V-CoP) is expected to be a potential way to ensure aviation safety. This study proposes a V-CoP concept and explores how humans and virtual assistants can effectively collaborate. A preliminary case study is conducted to explore a critical role of V-CoP, namely automated quick procedures searching, using the multimodal large language model (LLM). The LLM-enabled V-CoP integrates the pilot instruction and real-time cockpit instrumental data to prompt applicable aviation manuals and operation procedures. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis and effective retrieval of procedure information. The results showed that the LLM-enabled V-CoP achieved high accuracy in situational analysis (90.5%) and effective retrieval of procedure information (86.5%). The proposed V-CoP is expected to provide a foundation for future virtual intelligent assistant development, improve the performance of single pilots, and reduce the risk of human errors in aviation. |
| title | Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operations |
| topic | Human-Computer Interaction |
| url | https://arxiv.org/abs/2403.16645 |