Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2021
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2112.09259 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910258659590144 |
|---|---|
| author | Spini, Pietro Emilio |
| author_facet | Spini, Pietro Emilio |
| contents | This paper studies the robustness of estimated policy effects to changes in the distribution of covariates, a key determinant of the external validity of (quasi)-experimental results. I propose a novel robustness metric $δ^*$ which measures the smallest covariate shift needed to invalidate an empirical claim about the policy effect (e.g., $ATE > 0$). I estimate $δ^*$ via de-biased GMM, achieving a parametric rate of convergence while accommodating machine-learning estimators of treatment-effect heterogeneity (e.g., LASSO, random forests, neural networks). I develop benchmarking and calibration exercises to interpret the magnitude of $δ^*$. I illustrate these tools in an application to the Oregon Health Insurance Experiment. Researchers can report $δ^*$ alongside the point estimate and standard error as a third number gauging external validity under covariate shifts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2112_09259 |
| institution | arXiv |
| publishDate | 2021 |
| record_format | arxiv |
| spellingShingle | A Simple Measure of Robustness for External Validity under Covariate Shifts Spini, Pietro Emilio Econometrics This paper studies the robustness of estimated policy effects to changes in the distribution of covariates, a key determinant of the external validity of (quasi)-experimental results. I propose a novel robustness metric $δ^*$ which measures the smallest covariate shift needed to invalidate an empirical claim about the policy effect (e.g., $ATE > 0$). I estimate $δ^*$ via de-biased GMM, achieving a parametric rate of convergence while accommodating machine-learning estimators of treatment-effect heterogeneity (e.g., LASSO, random forests, neural networks). I develop benchmarking and calibration exercises to interpret the magnitude of $δ^*$. I illustrate these tools in an application to the Oregon Health Insurance Experiment. Researchers can report $δ^*$ alongside the point estimate and standard error as a third number gauging external validity under covariate shifts. |
| title | A Simple Measure of Robustness for External Validity under Covariate Shifts |
| topic | Econometrics |
| url | https://arxiv.org/abs/2112.09259 |