Monitoring and Observability of Machine Learning Systems: Current Practices and Gaps
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866915582440374272 |
|---|---|
| author | Leest, Joran Gerostathopoulos, Ilias Lago, Patricia Raibulet, Claudia |
| author_facet | Leest, Joran Gerostathopoulos, Ilias Lago, Patricia Raibulet, Claudia |
| contents | Production machine learning (ML) systems fail silently -- not with crashes, but through wrong decisions. While observability is recognized as critical for ML operations, there is a lack empirical evidence of what practitioners actually capture. This study presents empirical results on ML observability in practice through seven focus group sessions in several domains. We catalog the information practitioners systematically capture across ML systems and their environment and map how they use it to validate models, detect and diagnose faults, and explain observed degradations. Finally, we identify gaps in current practice and outline implications for tooling design and research to establish ML observability practices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_24142 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Monitoring and Observability of Machine Learning Systems: Current Practices and Gaps Leest, Joran Gerostathopoulos, Ilias Lago, Patricia Raibulet, Claudia Software Engineering Production machine learning (ML) systems fail silently -- not with crashes, but through wrong decisions. While observability is recognized as critical for ML operations, there is a lack empirical evidence of what practitioners actually capture. This study presents empirical results on ML observability in practice through seven focus group sessions in several domains. We catalog the information practitioners systematically capture across ML systems and their environment and map how they use it to validate models, detect and diagnose faults, and explain observed degradations. Finally, we identify gaps in current practice and outline implications for tooling design and research to establish ML observability practices. |
| title | Monitoring and Observability of Machine Learning Systems: Current Practices and Gaps |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2510.24142 |