Designing Latent Safety Filters using Pre-Trained Vision Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914044771827712 |
|---|---|
| author | Tabbara, Ihab Yang, Yuxuan Hamzeh, Ahmad Astafyev, Maxwell Sibai, Hussein |
| author_facet | Tabbara, Ihab Yang, Yuxuan Hamzeh, Ahmad Astafyev, Maxwell Sibai, Hussein |
| contents | Ensuring safety of vision-based control systems remains a major challenge hindering their deployment in critical settings. Safety filters have gained increased interest as effective tools for ensuring the safety of classical control systems, but their applications in vision-based control settings have so far been limited. Pre-trained vision models (PVRs) have been shown to be effective perception backbones for control in various robotics domains. In this paper, we are interested in examining their effectiveness when used for designing vision-based safety filters. We use them as backbones for classifiers defining failure sets, for Hamilton-Jacobi (HJ) reachability-based safety filters, and for latent world models. We discuss the trade-offs between training from scratch, fine-tuning, and freezing the PVRs when training the models they are backbones for. We also evaluate whether one of the PVRs is superior across all tasks, evaluate whether learned world models or Q-functions are better for switching decisions to safe policies, and discuss practical considerations for deploying these PVRs on resource-constrained devices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14758 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Designing Latent Safety Filters using Pre-Trained Vision Models Tabbara, Ihab Yang, Yuxuan Hamzeh, Ahmad Astafyev, Maxwell Sibai, Hussein Robotics Computer Vision and Pattern Recognition Machine Learning Systems and Control Ensuring safety of vision-based control systems remains a major challenge hindering their deployment in critical settings. Safety filters have gained increased interest as effective tools for ensuring the safety of classical control systems, but their applications in vision-based control settings have so far been limited. Pre-trained vision models (PVRs) have been shown to be effective perception backbones for control in various robotics domains. In this paper, we are interested in examining their effectiveness when used for designing vision-based safety filters. We use them as backbones for classifiers defining failure sets, for Hamilton-Jacobi (HJ) reachability-based safety filters, and for latent world models. We discuss the trade-offs between training from scratch, fine-tuning, and freezing the PVRs when training the models they are backbones for. We also evaluate whether one of the PVRs is superior across all tasks, evaluate whether learned world models or Q-functions are better for switching decisions to safe policies, and discuss practical considerations for deploying these PVRs on resource-constrained devices. |
| title | Designing Latent Safety Filters using Pre-Trained Vision Models |
| topic | Robotics Computer Vision and Pattern Recognition Machine Learning Systems and Control |
| url | https://arxiv.org/abs/2509.14758 |