Nonexperimental Designs and Program Evaluation

Sometimes implementing evaluation designs that involve random assignment or strong nonrandomized comparison group designs is not feasible. When this is the case, several other types of comparisons may offer credible, though not conclusive, evidence on program effects. This article describes three approaches—comparisons of the outcomes of the treatment group with a national sample, comparisons of the outcomes of program participants and nonparticipants, and dose-response analyses—and illustrates their use in the evaluation of the School-Based Adolescent Health Care Program. The findings suggest that if outcomes are measured before and after the intervention, comparisons of the treatment group outcomes to outcomes for a national sample may provide valid estimates of program effects. The other two types of comparisons produced implausible and unstable estimates of program effects. Because of the selection bias inherent in these two methods, researchers cannot count on being able to produce plausible estimates of program effects with such comparisons.