An approach for addressing the multiple testing problem in social policy impact evaluations

PZ Schochet - Evaluation Review, 2009 - journals.sagepub.com
Evaluation Review, 2009journals.sagepub.com
In social policy evaluations, the multiple testing problem occurs due to the many hypothesis
tests that are typically conducted across multiple outcomes and subgroups, which can lead
to spurious impact findings. This article discusses a framework for addressing this problem
that balances Types I and II errors. The framework involves specifying confirmatory and
exploratory analyses in study protocols, delineating confirmatory outcome domains,
conducting t tests on composite domain outcomes, and applying multiplicity corrections to …
In social policy evaluations, the multiple testing problem occurs due to the many hypothesis tests that are typically conducted across multiple outcomes and subgroups, which can lead to spurious impact findings. This article discusses a framework for addressing this problem that balances Types I and II errors. The framework involves specifying confirmatory and exploratory analyses in study protocols, delineating confirmatory outcome domains, conducting t tests on composite domain outcomes, and applying multiplicity corrections to composites across domains to obtain summative impact evidence. The article presents statistical background and discusses multiplicity issues for subgroup analyses, designs with multiple treatments, and reporting.
Sage Journals
以上显示的是最相近的搜索结果。 查看全部搜索结果