Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2405.05455 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910439542095872 |
|---|---|
| author | Renzullo, Joseph Reiter, Pemma Weimer, Westley Forrest, Stephanie |
| author_facet | Renzullo, Joseph Reiter, Pemma Weimer, Westley Forrest, Stephanie |
| contents | Machine learning (ML) now pervades the field of Automated Program Repair (APR). Algorithms deploy neural machine translation and large language models (LLMs) to generate software patches, among other tasks. But, there are important differences between these applications of ML and earlier work. Evaluations and comparisons must take care to ensure that results are valid and likely to generalize. A challenge is that the most popular APR evaluation benchmarks were not designed with ML techniques in mind. This is especially true for LLMs, whose large and often poorly-disclosed training datasets may include problems on which they are evaluated. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2405_05455 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Automated Program Repair: Emerging trends pose and expose problems for benchmarks Renzullo, Joseph Reiter, Pemma Weimer, Westley Forrest, Stephanie Software Engineering Machine Learning Machine learning (ML) now pervades the field of Automated Program Repair (APR). Algorithms deploy neural machine translation and large language models (LLMs) to generate software patches, among other tasks. But, there are important differences between these applications of ML and earlier work. Evaluations and comparisons must take care to ensure that results are valid and likely to generalize. A challenge is that the most popular APR evaluation benchmarks were not designed with ML techniques in mind. This is especially true for LLMs, whose large and often poorly-disclosed training datasets may include problems on which they are evaluated. |
| title | Automated Program Repair: Emerging trends pose and expose problems for benchmarks |
| topic | Software Engineering Machine Learning |
| url | https://arxiv.org/abs/2405.05455 |