Making Uncertainty Visible: Multiverse Analysis for Robust Computational Social Science
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913146268024832 |
|---|---|
| author | Linde, Maximilian Sun, Jun Balluff, Paul Radovanović, Danica Chan, Chung-hong |
| author_facet | Linde, Maximilian Sun, Jun Balluff, Paul Radovanović, Danica Chan, Chung-hong |
| contents | Through case studies, we demonstrate how multiverse analysis can strengthen the robustness and transparency of computational social science findings against alternative methodological decisions. We conduct multiverse analyses of three published social science studies that use the following computational methods: Bayesian analysis, network generative modeling, and machine learning with or without large language models. These methods are applied frequently in computational social science studies, yet entail a greater degree of arbitrariness in terms of methodological choices, or "researcher degrees of freedom." Our multiverse analyses reveal how the empirical findings in these studies vary as a function of various plausible decision combinations. Our three case studies also expose an often-ignored motivation for conducting multiverse analysis: Showing which methodological combinations lead to computational failure. These failed cases are usually not communicated in the published reports, even though these sophisticated computational methods have a much higher likelihood of failure. We end our paper with suggestions on how to find defensible decision combinations for multiverse analysis of computational social science studies and how to communicate multiverse analysis findings fairly. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2605_19745 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Making Uncertainty Visible: Multiverse Analysis for Robust Computational Social Science Linde, Maximilian Sun, Jun Balluff, Paul Radovanović, Danica Chan, Chung-hong Other Statistics Through case studies, we demonstrate how multiverse analysis can strengthen the robustness and transparency of computational social science findings against alternative methodological decisions. We conduct multiverse analyses of three published social science studies that use the following computational methods: Bayesian analysis, network generative modeling, and machine learning with or without large language models. These methods are applied frequently in computational social science studies, yet entail a greater degree of arbitrariness in terms of methodological choices, or "researcher degrees of freedom." Our multiverse analyses reveal how the empirical findings in these studies vary as a function of various plausible decision combinations. Our three case studies also expose an often-ignored motivation for conducting multiverse analysis: Showing which methodological combinations lead to computational failure. These failed cases are usually not communicated in the published reports, even though these sophisticated computational methods have a much higher likelihood of failure. We end our paper with suggestions on how to find defensible decision combinations for multiverse analysis of computational social science studies and how to communicate multiverse analysis findings fairly. |
| title | Making Uncertainty Visible: Multiverse Analysis for Robust Computational Social Science |
| topic | Other Statistics |
| url | https://arxiv.org/abs/2605.19745 |