BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908618143563776 |
|---|---|
| author | Sonwane, Atharv White, Isadora Lee, Hyunji Pereira, Matheus Caccia, Lucas Kim, Minseon Shi, Zhengyan Singh, Chinmay Sordoni, Alessandro Côté, Marc-Alexandre Yuan, Xingdi |
| author_facet | Sonwane, Atharv White, Isadora Lee, Hyunji Pereira, Matheus Caccia, Lucas Kim, Minseon Shi, Zhengyan Singh, Chinmay Sordoni, Alessandro Côté, Marc-Alexandre Yuan, Xingdi |
| contents | High quality bugs are key to training the next generation of language model based software engineering (SWE) agents. We introduce a novel method for synthetic generation of difficult and diverse bugs. Our method instructs SWE Agents to introduce a feature into the codebase whereby they may unintentionally break tests, resulting in bugs. Prior approaches often induce an out-of-distribution effect by generating bugs intentionally (e.g. by introducing local perturbation to existing code), which does not reflect realistic development processes. We perform qualitative analysis to demonstrate that our approach for generating bugs more closely reflects the patterns found in human-authored edits. Through extensive experiments, we demonstrate that our bugs provide more efficient training data for supervised fine-tuning, outperforming other bug datasets by 2% with half the training data (1.2k vs. 3k bugs). We train on our newly generated bugs in addition to existing bug datasets to get FrogBoss a state-of-the-art 32B parameter model on SWE-bench Verified with a pass@1 of 54.6% and FrogMini a state-of-the-art 14B model on SWE-bench Verified with a pass@1 of 45.3% on SWE-bench Verified averaged over three seeds. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_19898 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills Sonwane, Atharv White, Isadora Lee, Hyunji Pereira, Matheus Caccia, Lucas Kim, Minseon Shi, Zhengyan Singh, Chinmay Sordoni, Alessandro Côté, Marc-Alexandre Yuan, Xingdi Software Engineering Artificial Intelligence Computation and Language High quality bugs are key to training the next generation of language model based software engineering (SWE) agents. We introduce a novel method for synthetic generation of difficult and diverse bugs. Our method instructs SWE Agents to introduce a feature into the codebase whereby they may unintentionally break tests, resulting in bugs. Prior approaches often induce an out-of-distribution effect by generating bugs intentionally (e.g. by introducing local perturbation to existing code), which does not reflect realistic development processes. We perform qualitative analysis to demonstrate that our approach for generating bugs more closely reflects the patterns found in human-authored edits. Through extensive experiments, we demonstrate that our bugs provide more efficient training data for supervised fine-tuning, outperforming other bug datasets by 2% with half the training data (1.2k vs. 3k bugs). We train on our newly generated bugs in addition to existing bug datasets to get FrogBoss a state-of-the-art 32B parameter model on SWE-bench Verified with a pass@1 of 54.6% and FrogMini a state-of-the-art 14B model on SWE-bench Verified with a pass@1 of 45.3% on SWE-bench Verified averaged over three seeds. |
| title | BugPilot: Complex Bug Generation for Efficient Learning of SWE Skills |
| topic | Software Engineering Artificial Intelligence Computation and Language |
| url | https://arxiv.org/abs/2510.19898 |