Systematic Reviews of Evidence
Evaluation provides the evidence on the effectiveness of a program. However, the quality of evidence matters. In our systematic review and meta-analysis studies, we form a catalog of evidence-based practices to document
- the impacts of several implemented policies and programs targeting a specific outcome, and
- the key program elements, such as the program components, content, design, implementation, and challenges in implementation.
Evidence quality rating system
In order to identify best evidence-based practices, evaluation studies of each program can be rated in terms of evidence quality. The quality can be determined by the evaluation design (e.g., randomized trial, quasi-experimental), internal validity to determine causal effects (e.g., existence of comparison groups, equivalence of intervention and comparison groups at baseline), generalization of results to other populations and settings, reliability and validity of data on outcomes, and validity of statistical conclusions (e.g., statistical adjustment for selected measures, handling missing data) (for only the quantitative evaluation studies).
An evaluation study with a high rating means that the study is well designed to estimate the effectiveness of the intervention. A moderate rating means that the study is well designed but has some weaknesses. A low rating means that one cannot know if changes in outcomes are due to the intervention or other factors.