Human-in-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866908756662550528 |
|---|---|
| author | Carvell, Ben Thomas, Marc Pace, Andrew Dorney, Christopher De Ath, George Everson, Richard Pepper, Nick Keane, Adam Tomlinson, Samuel Cannon, Richard |
| author_facet | Carvell, Ben Thomas, Marc Pace, Andrew Dorney, Christopher De Ath, George Everson, Richard Pepper, Nick Keane, Adam Tomlinson, Samuel Cannon, Richard |
| contents | We present a rigorous, human-in-the-loop evaluation framework for assessing the performance of AI agents on the task of Air Traffic Control, grounded in a regulator-certified simulator-based curriculum used for training and testing real-world trainee controllers. By leveraging legally regulated assessments and involving expert human instructors in the evaluation process, our framework enables a more authentic and domain-accurate measurement of AI performance. This work addresses a critical gap in the existing literature: the frequent misalignment between academic representations of Air Traffic Control and the complexities of the actual operational environment. It also lays the foundations for effective future human-machine teaming paradigms by aligning machine performance with human assessment targets. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_04288 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Human-in-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework Carvell, Ben Thomas, Marc Pace, Andrew Dorney, Christopher De Ath, George Everson, Richard Pepper, Nick Keane, Adam Tomlinson, Samuel Cannon, Richard Human-Computer Interaction Machine Learning Multiagent Systems We present a rigorous, human-in-the-loop evaluation framework for assessing the performance of AI agents on the task of Air Traffic Control, grounded in a regulator-certified simulator-based curriculum used for training and testing real-world trainee controllers. By leveraging legally regulated assessments and involving expert human instructors in the evaluation process, our framework enables a more authentic and domain-accurate measurement of AI performance. This work addresses a critical gap in the existing literature: the frequent misalignment between academic representations of Air Traffic Control and the complexities of the actual operational environment. It also lays the foundations for effective future human-machine teaming paradigms by aligning machine performance with human assessment targets. |
| title | Human-in-the-Loop Testing of AI Agents for Air Traffic Control with a Regulated Assessment Framework |
| topic | Human-Computer Interaction Machine Learning Multiagent Systems |
| url | https://arxiv.org/abs/2601.04288 |