Agentic LMs: Hunting Down Test Smells
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914163770523648 |
|---|---|
| author | Melo, Rian Simões, Pedro Gheyi, Rohit d'Amorim, Marcelo Ribeiro, Márcio Soares, Gustavo Almeida, Eduardo Soares, Elvys |
| author_facet | Melo, Rian Simões, Pedro Gheyi, Rohit d'Amorim, Marcelo Ribeiro, Márcio Soares, Gustavo Almeida, Eduardo Soares, Elvys |
| contents | Test smells reduce test suite reliability and complicate maintenance. While many methods detect test smells, few support automated removal, and most rely on static analysis or machine learning. This study evaluates models with relatively small parameter counts - Llama-3.2-3B, Gemma-2-9B, DeepSeek-R1-14B, and Phi-4-14B - for their ability to detect and refactor test smells using agent-based workflows. We assess workflows with one, two, and four agents over 150 instances of 5 common smells from real-world Java projects. Our approach generalizes to Python, Golang, and JavaScript. All models detected nearly all instances, with Phi-4-14B achieving the best refactoring accuracy (pass@5 of 75.3%). Phi-4-14B with four-agents performed within 5% of proprietary LLMs (single-agent). Multi-agent setups outperformed single-agent ones in three of five smell types, though for Assertion Roulette, one agent sufficed. We submitted pull requests with Phi-4-14B-generated code to open-source projects and six were merged. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07277 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Agentic LMs: Hunting Down Test Smells Melo, Rian Simões, Pedro Gheyi, Rohit d'Amorim, Marcelo Ribeiro, Márcio Soares, Gustavo Almeida, Eduardo Soares, Elvys Software Engineering Test smells reduce test suite reliability and complicate maintenance. While many methods detect test smells, few support automated removal, and most rely on static analysis or machine learning. This study evaluates models with relatively small parameter counts - Llama-3.2-3B, Gemma-2-9B, DeepSeek-R1-14B, and Phi-4-14B - for their ability to detect and refactor test smells using agent-based workflows. We assess workflows with one, two, and four agents over 150 instances of 5 common smells from real-world Java projects. Our approach generalizes to Python, Golang, and JavaScript. All models detected nearly all instances, with Phi-4-14B achieving the best refactoring accuracy (pass@5 of 75.3%). Phi-4-14B with four-agents performed within 5% of proprietary LLMs (single-agent). Multi-agent setups outperformed single-agent ones in three of five smell types, though for Assertion Roulette, one agent sufficed. We submitted pull requests with Phi-4-14B-generated code to open-source projects and six were merged. |
| title | Agentic LMs: Hunting Down Test Smells |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2504.07277 |