A City upon a Hill: Casting Light on a Real Experimental Process
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915695166488576 |
|---|---|
| author | C., Efraín R. Fonseca López-Fernández, Marta Dieste, Oscar Juristo, Natalia |
| author_facet | C., Efraín R. Fonseca López-Fernández, Marta Dieste, Oscar Juristo, Natalia |
| contents | Context: The overall scientific community is proposing measures to improve the reproducibility and replicability of experiments. Reproducibility is relatively easy to achieve. However, replicability is considerably more complex in both the sciences and Empirical Software Engineering (ESE). Several strategies, e.g., replication packages and families of experiments, have been proposed to improve replication in ESE, with limited success. We wonder whether the failures are due to some mismatch, i.e., the researchers' needs are not satisfied by the proposed replication procedures.
Objectives: Find out how experimental researchers conduct \textit{experiments in practice}.
Methods: We carried out an ethnography study within a SE Research Group. Our main activity was to observe/approach the experimental researchers in their day-to-day settings for two years. Their preferred literature and experimental materials were studied. We used individual and group interviews to gain understanding and examine unclear topics in-depth.
Results: We have created conceptual and process models that represent how experimentation is really conducted in the Research Group. Models fit the community's procedures and terminology at a high level, but they become particular in their minute details.
Conclusion: The actual experimental process differs from textbooks in several points, namely: (1) Number and diversity of activities, (2) existence of different roles, (3) the granularity of the concepts used by the roles, and (4) the viewpoints that different sub-areas or families of experiments have about the overall process. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2108_12800 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | A City upon a Hill: Casting Light on a Real Experimental Process C., Efraín R. Fonseca López-Fernández, Marta Dieste, Oscar Juristo, Natalia Software Engineering Context: The overall scientific community is proposing measures to improve the reproducibility and replicability of experiments. Reproducibility is relatively easy to achieve. However, replicability is considerably more complex in both the sciences and Empirical Software Engineering (ESE). Several strategies, e.g., replication packages and families of experiments, have been proposed to improve replication in ESE, with limited success. We wonder whether the failures are due to some mismatch, i.e., the researchers' needs are not satisfied by the proposed replication procedures. Objectives: Find out how experimental researchers conduct \textit{experiments in practice}. Methods: We carried out an ethnography study within a SE Research Group. Our main activity was to observe/approach the experimental researchers in their day-to-day settings for two years. Their preferred literature and experimental materials were studied. We used individual and group interviews to gain understanding and examine unclear topics in-depth. Results: We have created conceptual and process models that represent how experimentation is really conducted in the Research Group. Models fit the community's procedures and terminology at a high level, but they become particular in their minute details. Conclusion: The actual experimental process differs from textbooks in several points, namely: (1) Number and diversity of activities, (2) existence of different roles, (3) the granularity of the concepts used by the roles, and (4) the viewpoints that different sub-areas or families of experiments have about the overall process. |
| title | A City upon a Hill: Casting Light on a Real Experimental Process |
| topic | Software Engineering |
| url | https://arxiv.org/abs/2108.12800 |