Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | https://arxiv.org/abs/2411.18714 |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911333105008640 |
|---|---|
| author | Kenny, Eoin M. Dharmavaram, Akshay Lee, Sang Uk Phan-Minh, Tung Rajesh, Shreyas Hu, Yunqing Major, Laura Tomov, Momchil S. Shah, Julie A. |
| author_facet | Kenny, Eoin M. Dharmavaram, Akshay Lee, Sang Uk Phan-Minh, Tung Rajesh, Shreyas Hu, Yunqing Major, Laura Tomov, Momchil S. Shah, Julie A. |
| contents | Self-driving cars increasingly rely on deep neural networks to achieve human-like driving. The opacity of such black-box planners makes it challenging for the human behind the wheel to accurately anticipate when they will fail, with potentially catastrophic consequences. While research into interpreting these systems has surged, most of it is confined to simulations or toy setups due to the difficulty of real-world deployment, leaving the practical utility of such techniques unknown. Here, we introduce the Concept-Wrapper Network (CW-Net), a method for explaining the behavior of machine-learning-based planners by grounding their reasoning in human-interpretable concepts. We deploy CW-Net on a real self-driving car and show that the resulting explanations improve the human driver's mental model of the car, allowing them to better predict its behavior. To our knowledge, this is the first demonstration that explainable deep learning integrated into self-driving cars can be both understandable and useful in a realistic deployment setting. CW-Net accomplishes this level of intelligibility while providing explanations which are causally faithful and do not sacrifice driving performance. Overall, our study establishes a general pathway to interpretability for autonomous agents by way of concept-based explanations, which could help make them more transparent and safe. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2411_18714 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explainable deep learning improves human mental models of self-driving cars Kenny, Eoin M. Dharmavaram, Akshay Lee, Sang Uk Phan-Minh, Tung Rajesh, Shreyas Hu, Yunqing Major, Laura Tomov, Momchil S. Shah, Julie A. Robotics Artificial Intelligence Machine Learning Self-driving cars increasingly rely on deep neural networks to achieve human-like driving. The opacity of such black-box planners makes it challenging for the human behind the wheel to accurately anticipate when they will fail, with potentially catastrophic consequences. While research into interpreting these systems has surged, most of it is confined to simulations or toy setups due to the difficulty of real-world deployment, leaving the practical utility of such techniques unknown. Here, we introduce the Concept-Wrapper Network (CW-Net), a method for explaining the behavior of machine-learning-based planners by grounding their reasoning in human-interpretable concepts. We deploy CW-Net on a real self-driving car and show that the resulting explanations improve the human driver's mental model of the car, allowing them to better predict its behavior. To our knowledge, this is the first demonstration that explainable deep learning integrated into self-driving cars can be both understandable and useful in a realistic deployment setting. CW-Net accomplishes this level of intelligibility while providing explanations which are causally faithful and do not sacrifice driving performance. Overall, our study establishes a general pathway to interpretability for autonomous agents by way of concept-based explanations, which could help make them more transparent and safe. |
| title | Explainable deep learning improves human mental models of self-driving cars |
| topic | Robotics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2411.18714 |