ViBR: Automated Bug Replay from Video-based Reports using Vision-Language Models
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866911613867524096 |
|---|---|
| author | Feng, Sidong Wang, Dingbang Tomic, Nikola Yu, Tingting Aleti, Aldeida Chen, Chunyang |
| author_facet | Feng, Sidong Wang, Dingbang Tomic, Nikola Yu, Tingting Aleti, Aldeida Chen, Chunyang |
| contents | Bug reports play a critical role in software maintenance by helping users convey encountered issues to developers. Recently, GUI screen capture videos have gained popularity as a bug reporting artifact due to their ease of use and ability to retain rich contextual information. However, automatically reproducing bugs from such recordings remains a significant challenge. Existing methods often rely on fragile image-processing heuristics, explicit touch indicators, or pre-constructed UI transition graphs, which require non-trivial instrumentation and app-specific setup. This paper presents ViBR, a lightweight and fully automated approach that reproduces bugs directly from GUI recordings. Specifically, ViBR combines CLIP-based embedding similarity for action boundary segmentation with Vision-Language Models (VLMs) for region-aware GUI state comparison and guided bug replay. Experimental results show that ViBR successfully reproduces 72% of bug recordings, significantly outperforming state-of-the-art baselines and ablation variants. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_19905 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | ViBR: Automated Bug Replay from Video-based Reports using Vision-Language Models Feng, Sidong Wang, Dingbang Tomic, Nikola Yu, Tingting Aleti, Aldeida Chen, Chunyang Software Engineering Bug reports play a critical role in software maintenance by helping users convey encountered issues to developers. Recently, GUI screen capture videos have gained popularity as a bug reporting artifact due to their ease of use and ability to retain rich contextual information. However, automatically reproducing bugs from such recordings remains a significant challenge. Existing methods often rely on fragile image-processing heuristics, explicit touch indicators, or pre-constructed UI transition graphs, which require non-trivial instrumentation and app-specific setup. This paper presents ViBR, a lightweight and fully automated approach that reproduces bugs directly from GUI recordings. Specifically, ViBR combines CLIP-based embedding similarity for action boundary segmentation with Vision-Language Models (VLMs) for region-aware GUI state comparison and guided bug replay. Experimental results show that ViBR successfully reproduces 72% of bug recordings, significantly outperforming state-of-the-art baselines and ablation variants. |
| title | ViBR: Automated Bug Replay from Video-based Reports using Vision-Language Models |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2604.19905 |