Evaluating services demonstration programs: A multistage approach

This paper describes the rationale for and an example of a multistage approach to evaluating services demonstration programs that takes account of the important design limitations often found in such programs. These limitations may include one or more of the following: (a) the lack of control or comparison groups — i.e., there is no requirement for experimental or quasi-experimental design; (b) the lack of a common intervention — i.e., the demonstrations involve multiple projects, each of which is implementing a different intervention; and (c) the lack of common data collection structure across projects. We view the evaluation of services demonstration programs as one step in the broader process through which new interventions are conceived, tested, and ultimately disseminated. We describe an approach to the evaluation of services demonstrations that provides an empirical basis for identifying those projects that appear “promising” (i.e., appear to be fulfilling the demonstration's objectives), and then describes a subset of promising projects in detail. Findings from such an evaluation can provide the basis for moving into the next phase, in which the effectiveness of one or more promising models is tested experimentally. As an example of how this approach can be applied, we describe the design of the National Evaluation of Models for Linking Drug Abuse Treatment and Primary Care, the evaluation of a federally-funded services demonstration that was aimed at examining alternative strategies for improving the linkage between the drug abuse treatment and primary care systems.