AutoSpec: Automated Generation of Neural Network Specifications
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914986632151040 |
|---|---|
| author | Jin, Shuowei Yan, Francis Y. Tan, Cheng Kalia, Anuj Foukas, Xenofon Mao, Z. Morley |
| author_facet | Jin, Shuowei Yan, Francis Y. Tan, Cheng Kalia, Anuj Foukas, Xenofon Mao, Z. Morley |
| contents | The increasing adoption of neural networks in learning-augmented systems highlights the importance of model safety and robustness, particularly in safety-critical domains. Despite progress in the formal verification of neural networks, current practices require users to manually define model specifications -- properties that dictate expected model behavior in various scenarios. This manual process, however, is prone to human error, limited in scope, and time-consuming. In this paper, we introduce AutoSpec, the first framework to automatically generate comprehensive and accurate specifications for neural networks in learning-augmented systems. We also propose the first set of metrics for assessing the accuracy and coverage of model specifications, establishing a benchmark for future comparisons. Our evaluation across four distinct applications shows that AutoSpec outperforms human-defined specifications as well as two baseline approaches introduced in this study. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_10897 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | AutoSpec: Automated Generation of Neural Network Specifications Jin, Shuowei Yan, Francis Y. Tan, Cheng Kalia, Anuj Foukas, Xenofon Mao, Z. Morley Machine Learning Software Engineering The increasing adoption of neural networks in learning-augmented systems highlights the importance of model safety and robustness, particularly in safety-critical domains. Despite progress in the formal verification of neural networks, current practices require users to manually define model specifications -- properties that dictate expected model behavior in various scenarios. This manual process, however, is prone to human error, limited in scope, and time-consuming. In this paper, we introduce AutoSpec, the first framework to automatically generate comprehensive and accurate specifications for neural networks in learning-augmented systems. We also propose the first set of metrics for assessing the accuracy and coverage of model specifications, establishing a benchmark for future comparisons. Our evaluation across four distinct applications shows that AutoSpec outperforms human-defined specifications as well as two baseline approaches introduced in this study. |
| title | AutoSpec: Automated Generation of Neural Network Specifications |
| topic | Machine Learning Software Engineering |
| url | https://arxiv.org/abs/2409.10897 |