Robust and efficient multiple-unit switchback experimentation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909649268113408 |
|---|---|
| author | Missault, Paul Masoero, Lorenzo Delbé, Christian Richardson, Thomas Imbens, Guido |
| author_facet | Missault, Paul Masoero, Lorenzo Delbé, Christian Richardson, Thomas Imbens, Guido |
| contents | User-randomized A/B testing has emerged as the gold standard for online experimentation. However, when this kind of approach is not feasible due to legal, ethical or practical considerations, experimenters have to consider alternatives like item-randomization. Item-randomization is often met with skepticism due to its poor empirical performance. To fill this gap, in this paper we introduce a novel and rich class of experimental designs, "Regular Balanced Switchback Designs" (RBSDs). At their core, RBSDs work by randomly changing treatment assignments over both time and items. After establishing the properties of our designs in a potential outcomes framework, characterizing assumptions and conditions under which corresponding estimators are resilient to the presence of carryover effects, we show empirically via both realistic simulations and real e-commerce data that RBSDs systematically outperform standard item-randomized and non-balanced switchback approaches by yielding much more accurate estimates of the causal effects of interest without incurring any additional bias. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_12654 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Robust and efficient multiple-unit switchback experimentation Missault, Paul Masoero, Lorenzo Delbé, Christian Richardson, Thomas Imbens, Guido Methodology Applications User-randomized A/B testing has emerged as the gold standard for online experimentation. However, when this kind of approach is not feasible due to legal, ethical or practical considerations, experimenters have to consider alternatives like item-randomization. Item-randomization is often met with skepticism due to its poor empirical performance. To fill this gap, in this paper we introduce a novel and rich class of experimental designs, "Regular Balanced Switchback Designs" (RBSDs). At their core, RBSDs work by randomly changing treatment assignments over both time and items. After establishing the properties of our designs in a potential outcomes framework, characterizing assumptions and conditions under which corresponding estimators are resilient to the presence of carryover effects, we show empirically via both realistic simulations and real e-commerce data that RBSDs systematically outperform standard item-randomized and non-balanced switchback approaches by yielding much more accurate estimates of the causal effects of interest without incurring any additional bias. |
| title | Robust and efficient multiple-unit switchback experimentation |
| topic | Methodology Applications |
| url | https://arxiv.org/abs/2506.12654 |