What Do Contribution Guidelines Say About Software Testing?
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917244262416384 |
|---|---|
| author | Falcucci, Bruna Gomide, Felipe Hora, Andre |
| author_facet | Falcucci, Bruna Gomide, Felipe Hora, Andre |
| contents | Software testing plays a crucial role in the contribution process of open-source projects. For example, contributions introducing new features are expected to include tests, and contributions with tests are more likely to be accepted. Although most real-world projects require contributors to write tests, the specific testing practices communicated to contributors remain unclear. In this paper, we present an empirical study to understand better how software testing is approached in contribution guidelines. We analyze the guidelines of 200 Python and JavaScript open-source software projects. We find that 78\% of the projects include some form of test documentation for contributors. Test documentation is located in multiple sources, including \texttt{CONTRIBUTING} files (58\%), external documentation (24\%), and \texttt{README} files (8\%). Furthermore, test documentation commonly explains how to run tests (83.5\%), but less often provides guidance on how to write tests (37\%). It frequently covers unit tests (71\%), but rarely addresses integration (20.5\%) and end-to-end tests (15.5\%). Other key testing aspects are also less frequently discussed: test coverage (25.5\%) and mocking (9.5\%). We conclude by discussing implications and future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_02966 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | What Do Contribution Guidelines Say About Software Testing? Falcucci, Bruna Gomide, Felipe Hora, Andre Software Engineering Software testing plays a crucial role in the contribution process of open-source projects. For example, contributions introducing new features are expected to include tests, and contributions with tests are more likely to be accepted. Although most real-world projects require contributors to write tests, the specific testing practices communicated to contributors remain unclear. In this paper, we present an empirical study to understand better how software testing is approached in contribution guidelines. We analyze the guidelines of 200 Python and JavaScript open-source software projects. We find that 78\% of the projects include some form of test documentation for contributors. Test documentation is located in multiple sources, including \texttt{CONTRIBUTING} files (58\%), external documentation (24\%), and \texttt{README} files (8\%). Furthermore, test documentation commonly explains how to run tests (83.5\%), but less often provides guidance on how to write tests (37\%). It frequently covers unit tests (71\%), but rarely addresses integration (20.5\%) and end-to-end tests (15.5\%). Other key testing aspects are also less frequently discussed: test coverage (25.5\%) and mocking (9.5\%). We conclude by discussing implications and future research. |
| title | What Do Contribution Guidelines Say About Software Testing? |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2602.02966 |