Best of Three Worlds: Adaptive Experimentation for Digital Marketing in Practice
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910608816865280 |
|---|---|
| author | Fiez, Tanner Nassif, Houssam Chen, Yu-Cheng Gamez, Sergio Jain, Lalit |
| author_facet | Fiez, Tanner Nassif, Houssam Chen, Yu-Cheng Gamez, Sergio Jain, Lalit |
| contents | Adaptive experimental design (AED) methods are increasingly being used in industry as a tool to boost testing throughput or reduce experimentation cost relative to traditional A/B/N testing methods. However, the behavior and guarantees of such methods are not well-understood beyond idealized stationary settings. This paper shares lessons learned regarding the challenges of naively using AED systems in industrial settings where non-stationarity is prevalent, while also providing perspectives on the proper objectives and system specifications in such settings. We developed an AED framework for counterfactual inference based on these experiences, and tested it in a commercial environment. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_10870 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Best of Three Worlds: Adaptive Experimentation for Digital Marketing in Practice Fiez, Tanner Nassif, Houssam Chen, Yu-Cheng Gamez, Sergio Jain, Lalit Machine Learning Methodology Adaptive experimental design (AED) methods are increasingly being used in industry as a tool to boost testing throughput or reduce experimentation cost relative to traditional A/B/N testing methods. However, the behavior and guarantees of such methods are not well-understood beyond idealized stationary settings. This paper shares lessons learned regarding the challenges of naively using AED systems in industrial settings where non-stationarity is prevalent, while also providing perspectives on the proper objectives and system specifications in such settings. We developed an AED framework for counterfactual inference based on these experiences, and tested it in a commercial environment. |
| title | Best of Three Worlds: Adaptive Experimentation for Digital Marketing in Practice |
| topic | Machine Learning Methodology |
| url | https://arxiv.org/abs/2402.10870 |