Choosing an analytic approach: Key study design considerations in state policy evaluation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908603985690624 |
|---|---|
| author | Stone, Elizabeth M. Schuler, Megan S. Stuart, Elizabeth A. Rubinstein, Max Griswold, Max Stein, Bradley D. Griffin, Beth Ann |
| author_facet | Stone, Elizabeth M. Schuler, Megan S. Stuart, Elizabeth A. Rubinstein, Max Griswold, Max Stein, Bradley D. Griffin, Beth Ann |
| contents | This paper reviews and details methods for state policy evaluation to guide selection of a research approach based on evaluation setting and available data. We highlight key design considerations for an analysis, including treatment and control group selection, timing of policy adoption, expected effect heterogeneity, and data considerations. We then provide an overview of analytic approaches and differentiate between methods based on evaluation context, such as settings with no control units, a single treated unit, multiple treated units, or with multiple treatment cohorts. Methods discussed include interrupted time series models, difference-in-differences estimators, autoregressive models, and synthetic control methods, along with method extensions which address issues like staggered policy adoption and heterogenous treatment effects. We end with an illustrative example, applying the developed framework to evaluate the impacts of state-level naloxone standing order policies on overdose rates. Overall, we provide researchers with an approach for deciding on methods for state policy evaluations, which can be used to select study designs and inform methodological choices. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_03609 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Choosing an analytic approach: Key study design considerations in state policy evaluation Stone, Elizabeth M. Schuler, Megan S. Stuart, Elizabeth A. Rubinstein, Max Griswold, Max Stein, Bradley D. Griffin, Beth Ann Methodology This paper reviews and details methods for state policy evaluation to guide selection of a research approach based on evaluation setting and available data. We highlight key design considerations for an analysis, including treatment and control group selection, timing of policy adoption, expected effect heterogeneity, and data considerations. We then provide an overview of analytic approaches and differentiate between methods based on evaluation context, such as settings with no control units, a single treated unit, multiple treated units, or with multiple treatment cohorts. Methods discussed include interrupted time series models, difference-in-differences estimators, autoregressive models, and synthetic control methods, along with method extensions which address issues like staggered policy adoption and heterogenous treatment effects. We end with an illustrative example, applying the developed framework to evaluate the impacts of state-level naloxone standing order policies on overdose rates. Overall, we provide researchers with an approach for deciding on methods for state policy evaluations, which can be used to select study designs and inform methodological choices. |
| title | Choosing an analytic approach: Key study design considerations in state policy evaluation |
| topic | Methodology |
| url | https://arxiv.org/abs/2504.03609 |