Evaluating Organizational Effectiveness: A New Strategy to Leverage Multisite Randomized Trials for Valid Assessment
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866914398914740224 |
|---|---|
| author | Hong, Guanglei Deutsch, Jonah Kress, Peter Trinidad, Jose Eos Xu, Zhengyan |
| author_facet | Hong, Guanglei Deutsch, Jonah Kress, Peter Trinidad, Jose Eos Xu, Zhengyan |
| contents | Determining which organizations are more effective in implementing an intervention program is essential for theoretically and empirically characterizing exemplary practice and for intervening to enhance the capacity of ineffective ones. Yet sites differ in their local ecological conditions including client composition, alternative programs, and community context. Applying the causal inference framework, this study proposes a formal mathematical definition for the local relative effectiveness of an organization attributable solely to malleable organizational practice. Capitalizing on multisite randomized trials, the identification leverages observed control group outcomes that capture some of the confounding impacts of otherwise unmeasured contextual variation. We propose a two-step mixed-effects modeling (2SME) procedure that adjusts for pre-existing between-site variation. A series of Monte Carlo simulations reveals its superior performance in comparison with conventional methods. We apply the new strategy to an evaluation of Job Corps centers nationwide serving disadvantaged youths. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_18360 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Evaluating Organizational Effectiveness: A New Strategy to Leverage Multisite Randomized Trials for Valid Assessment Hong, Guanglei Deutsch, Jonah Kress, Peter Trinidad, Jose Eos Xu, Zhengyan Applications Determining which organizations are more effective in implementing an intervention program is essential for theoretically and empirically characterizing exemplary practice and for intervening to enhance the capacity of ineffective ones. Yet sites differ in their local ecological conditions including client composition, alternative programs, and community context. Applying the causal inference framework, this study proposes a formal mathematical definition for the local relative effectiveness of an organization attributable solely to malleable organizational practice. Capitalizing on multisite randomized trials, the identification leverages observed control group outcomes that capture some of the confounding impacts of otherwise unmeasured contextual variation. We propose a two-step mixed-effects modeling (2SME) procedure that adjusts for pre-existing between-site variation. A series of Monte Carlo simulations reveals its superior performance in comparison with conventional methods. We apply the new strategy to an evaluation of Job Corps centers nationwide serving disadvantaged youths. |
| title | Evaluating Organizational Effectiveness: A New Strategy to Leverage Multisite Randomized Trials for Valid Assessment |
| topic | Applications |
| url | https://arxiv.org/abs/2407.18360 |