Twenty-five strategies for improving the design, implementation and analysis of health services research related to alcohol and other drug abuse treatment

While some aspects of addiction can be studied in laboratory or controlled settings, the study of long‐term recovery management and the health services that support it requires going out into the community and dealing with populations and systems that are much more diverse and less under our control. This in turn raises many methodological challenges for the health service researchers studying alcohol and other drug abuse treatment. This paper identifies some of these challenges related to the design, measurement, implementation and effectiveness of health services research. It then recommends 25 strategies (and key primers) for addressing them: (1) identifying in advance all stakeholders and issues; (2) developing conceptual models of intervention and context; (3) identifying the population to whom the conclusions will be generalized; (4) matching the research design to the question; (5) conducting randomized experiments only when appropriate and necessary; (6) balancing methodological and treatment concerns; (7) prioritizing analysis plans and increasing design sensitivity, (8) combining qualitative and quantitative methods; (9) identifying the four basic types of measures needed; (10) identifying and using standardized measures; (11) carefully balancing measurement selection and modification; (12) developing and evaluating modified and new measures when necessary; (13) identifying and tracking major clinical subgroups; (14) measuring and analyzing the actual pattern of services received; (15) incorporating implementation checks into the design; (16) incorporating baseline measures into the intervention; (17) monitoring implementation and dosage as a form of quality assurance; (18) developing procedures early to facilitate tracking and follow‐up of study participants; (19) using more appropriate representations of the actual experiment; (20) using appropriate and sensitive standard deviation terms; (21) partialing out variance due to design or known sources prior to estimating experimental effect sizes; (22) using dimensional, interval and ratio measures to increase sensitivity to change; (23) using path or structural equation models; (24) integrating qualitative and quantitative analysis into reporting; and (25) using quasi‐experiments, economic or organizational studies to answer other likely policy questions. Most of these strategies have been tried and tested in this and other areas, but are not widely used. Improving the state of the art of health s