Automated Unit Test Improvement using Large Language Models at Meta
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917590136258560 |
|---|---|
| author | Alshahwan, Nadia Chheda, Jubin Finegenova, Anastasia Gokkaya, Beliz Harman, Mark Harper, Inna Marginean, Alexandru Sengupta, Shubho Wang, Eddy |
| author_facet | Alshahwan, Nadia Chheda, Jubin Finegenova, Anastasia Gokkaya, Beliz Harman, Mark Harper, Inna Marginean, Alexandru Sengupta, Shubho Wang, Eddy |
| contents | This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM's test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta's Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_09171 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Automated Unit Test Improvement using Large Language Models at Meta Alshahwan, Nadia Chheda, Jubin Finegenova, Anastasia Gokkaya, Beliz Harman, Mark Harper, Inna Marginean, Alexandru Sengupta, Shubho Wang, Eddy Software Engineering This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM's test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta's Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement. |
| title | Automated Unit Test Improvement using Large Language Models at Meta |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2402.09171 |