What Happened in This Pipeline? Diffing Build Logs with CiDiff
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916707571859456 |
|---|---|
| author | Hubner, Nicolas Falleri, Jean-Rémy Uricaru, Raluca Degueule, Thomas Durieux, Thomas |
| author_facet | Hubner, Nicolas Falleri, Jean-Rémy Uricaru, Raluca Degueule, Thomas Durieux, Thomas |
| contents | Continuous integration (CI) is widely used by developers to ensure the quality and reliability of their software projects. However, diagnosing a CI regression is a tedious process that involves the manual analysis of lengthy build logs. In this paper, we explore how textual differencing can support the debugging of CI regressions. As off-the-shelf diff algorithms produce suboptimal results, in this work we introduce a new diff algorithm specifically tailored to build logs called CiDiff. We evaluate CiDiff against several baselines on a novel dataset of 17 906 CI regressions, performing an accuracy study, a quantitative study and a user-study. Notably, our algorithm reduces the number of lines to inspect by about 60 % in the median case, with reasonable overhead compared to the state-of-practice LCS-diff. Finally, our algorithm is preferred by the majority of participants in 70 % of the regression cases, whereas LCS-diff is preferred in only 5 % of the cases. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_18182 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | What Happened in This Pipeline? Diffing Build Logs with CiDiff Hubner, Nicolas Falleri, Jean-Rémy Uricaru, Raluca Degueule, Thomas Durieux, Thomas Software Engineering Continuous integration (CI) is widely used by developers to ensure the quality and reliability of their software projects. However, diagnosing a CI regression is a tedious process that involves the manual analysis of lengthy build logs. In this paper, we explore how textual differencing can support the debugging of CI regressions. As off-the-shelf diff algorithms produce suboptimal results, in this work we introduce a new diff algorithm specifically tailored to build logs called CiDiff. We evaluate CiDiff against several baselines on a novel dataset of 17 906 CI regressions, performing an accuracy study, a quantitative study and a user-study. Notably, our algorithm reduces the number of lines to inspect by about 60 % in the median case, with reasonable overhead compared to the state-of-practice LCS-diff. Finally, our algorithm is preferred by the majority of participants in 70 % of the regression cases, whereas LCS-diff is preferred in only 5 % of the cases. |
| title | What Happened in This Pipeline? Diffing Build Logs with CiDiff |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2504.18182 |