Anytime-Valid Linear Models and Regression Adjusted Causal Inference in Randomized Experiments
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2022
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915374144946176 |
|---|---|
| author | Lindon, Michael Ham, Dae Woong Tingley, Martin Bojinov, Iavor |
| author_facet | Lindon, Michael Ham, Dae Woong Tingley, Martin Bojinov, Iavor |
| contents | Linear models are foundational tools in statistics and ubiquitous across the applied sciences. However, conventional statistical inference -- such as $t$-tests and $F$-tests -- are only valid at fixed sample sizes, making them unsuitable for sequential settings such as online A/B testing. We develop an anytime-valid theory of inference for the linear model, introducing sequential analogues of classical tests and confidence sets that provide Type-I error control and coverage guarantees uniformly over all sample sizes. Our construction is based on likelihood ratios of invariantly sufficient statistics, yielding simple closed-form expressions of ordinary least squares estimators and standard errors. The resulting tests are optimal in the GROW/REGROW sense for both frequentist and Bayesian alternative hypotheses. We then relax the linear model assumptions to provide heteroskedasticity-robust asymptotic sequential tests and confidence sequences, which enable sequential regression-adjusted inference for causal estimands in randomized controlled experiments. This formally allows experiments to be continuously monitored for significance, stopped early, and safeguards against statistical malpractices in data collection. We demonstrate the practical utility of our approach through simulations and applications to real A/B test data from Netflix. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2210_08589 |
| institution | arXiv |
| publishDate | 2022 |
| record_format | arxiv |
| spellingShingle | Anytime-Valid Linear Models and Regression Adjusted Causal Inference in Randomized Experiments Lindon, Michael Ham, Dae Woong Tingley, Martin Bojinov, Iavor Methodology Statistics Theory Linear models are foundational tools in statistics and ubiquitous across the applied sciences. However, conventional statistical inference -- such as $t$-tests and $F$-tests -- are only valid at fixed sample sizes, making them unsuitable for sequential settings such as online A/B testing. We develop an anytime-valid theory of inference for the linear model, introducing sequential analogues of classical tests and confidence sets that provide Type-I error control and coverage guarantees uniformly over all sample sizes. Our construction is based on likelihood ratios of invariantly sufficient statistics, yielding simple closed-form expressions of ordinary least squares estimators and standard errors. The resulting tests are optimal in the GROW/REGROW sense for both frequentist and Bayesian alternative hypotheses. We then relax the linear model assumptions to provide heteroskedasticity-robust asymptotic sequential tests and confidence sequences, which enable sequential regression-adjusted inference for causal estimands in randomized controlled experiments. This formally allows experiments to be continuously monitored for significance, stopped early, and safeguards against statistical malpractices in data collection. We demonstrate the practical utility of our approach through simulations and applications to real A/B test data from Netflix. |
| title | Anytime-Valid Linear Models and Regression Adjusted Causal Inference in Randomized Experiments |
| topic | Methodology Statistics Theory |
| url | https://arxiv.org/abs/2210.08589 |