Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2406.08575 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913389767294976 |
|---|---|
| author | Brower-Sinning, Rachel Lewis, Grace A. Echeverría, Sebastían Ozkaya, Ipek |
| author_facet | Brower-Sinning, Rachel Lewis, Grace A. Echeverría, Sebastían Ozkaya, Ipek |
| contents | Testing of machine learning (ML) models is a known challenge identified by researchers and practitioners alike. Unfortunately, current practice for ML model testing prioritizes testing for model performance, while often neglecting the requirements and constraints of the ML-enabled system that integrates the model. This limited view of testing leads to failures during integration, deployment, and operations, contributing to the difficulties of moving models from development to production. This paper presents an approach based on quality attribute (QA) scenarios to elicit and define system- and model-relevant test cases for ML models. The QA-based approach described in this paper has been integrated into MLTE, a process and tool to support ML model test and evaluation. Feedback from users of MLTE highlights its effectiveness in testing beyond model performance and identifying failures early in the development process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_08575 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Using Quality Attribute Scenarios for ML Model Test Case Generation Brower-Sinning, Rachel Lewis, Grace A. Echeverría, Sebastían Ozkaya, Ipek Software Engineering Artificial Intelligence Machine Learning Testing of machine learning (ML) models is a known challenge identified by researchers and practitioners alike. Unfortunately, current practice for ML model testing prioritizes testing for model performance, while often neglecting the requirements and constraints of the ML-enabled system that integrates the model. This limited view of testing leads to failures during integration, deployment, and operations, contributing to the difficulties of moving models from development to production. This paper presents an approach based on quality attribute (QA) scenarios to elicit and define system- and model-relevant test cases for ML models. The QA-based approach described in this paper has been integrated into MLTE, a process and tool to support ML model test and evaluation. Feedback from users of MLTE highlights its effectiveness in testing beyond model performance and identifying failures early in the development process. |
| title | Using Quality Attribute Scenarios for ML Model Test Case Generation |
| topic | Software Engineering Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2406.08575 |