Conformal Safety Shielding for Imperfect-Perception Agents
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915412723105792 |
|---|---|
| author | Scarbro, William Imrie, Calum Yaman, Sinem Getir Fatehi, Kavan Pasareanu, Corina S. Calinescu, Radu Mangal, Ravi |
| author_facet | Scarbro, William Imrie, Calum Yaman, Sinem Getir Fatehi, Kavan Pasareanu, Corina S. Calinescu, Radu Mangal, Ravi |
| contents | We consider the problem of safe control in discrete autonomous agents that use learned components for imperfect perception (or more generally, state estimation) from high-dimensional observations. We propose a shield construction that provides run-time safety guarantees under perception errors by restricting the actions available to an agent, modeled as a Markov decision process, as a function of the state estimates. Our construction uses conformal prediction for the perception component, which guarantees that for each observation, the predicted set of estimates includes the actual state with a user-specified probability. The shield allows an action only if it is allowed for all the estimates in the predicted set, resulting in local safety. We also articulate and prove a global safety property of existing shield constructions for perfect-perception agents bounding the probability of reaching unsafe states if the agent always chooses actions prescribed by the shield. We illustrate our approach with a case-study of an experimental autonomous system that guides airplanes on taxiways using high-dimensional perception DNNs. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_17275 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Conformal Safety Shielding for Imperfect-Perception Agents Scarbro, William Imrie, Calum Yaman, Sinem Getir Fatehi, Kavan Pasareanu, Corina S. Calinescu, Radu Mangal, Ravi Systems and Control Artificial Intelligence Machine Learning We consider the problem of safe control in discrete autonomous agents that use learned components for imperfect perception (or more generally, state estimation) from high-dimensional observations. We propose a shield construction that provides run-time safety guarantees under perception errors by restricting the actions available to an agent, modeled as a Markov decision process, as a function of the state estimates. Our construction uses conformal prediction for the perception component, which guarantees that for each observation, the predicted set of estimates includes the actual state with a user-specified probability. The shield allows an action only if it is allowed for all the estimates in the predicted set, resulting in local safety. We also articulate and prove a global safety property of existing shield constructions for perfect-perception agents bounding the probability of reaching unsafe states if the agent always chooses actions prescribed by the shield. We illustrate our approach with a case-study of an experimental autonomous system that guides airplanes on taxiways using high-dimensional perception DNNs. |
| title | Conformal Safety Shielding for Imperfect-Perception Agents |
| topic | Systems and Control Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2506.17275 |