Quantitative impact of including consumers in the scientific review of breast cancer research proposals

To evaluate the impact of having breast cancer survivors with advocacy experience (consumers) participate as voting members of scientific review panels for proposals on breast cancer research. As major stakeholders, patients and other consumer advocates sought inclusion in all decision-making processes affecting funding of disease-targeted research. Cross-sectional analysis of assigned proposal scores ranging from 5.0 (acceptable) to 1.0 (outstanding); before (prepanel) and after (postpanel) opinion questionnaires. Forty-six panels reviewed 2206 proposals for the Fiscal Year 1995 Department of Defense (DOD) Breast Cancer Research Program. Analyses were limited to the 42 panels scheduled to meet on site and the 2190 proposals scored by both participant groups. There were 85 consumers and 638 scientists. The main outcome measures were proposal merit scores (raw, overall, and participant-specific means) and opinions concerning perceived benefits and drawbacks of consumer involvement. In general, the voting patterns of consumers were similar to those of scientists. Final proposal scores were the same as those that would have been obtained without consumer voting for 76.2% of the proposals, more favorable for 15.2% of the proposals, and less favorable for 8.6% of the proposals. For all but 13 proposals, the difference was +/-0.1. Prepanel opinions regarding consumer involvement were generally positive. Prepanel and postpanel comparisons almost always showed that significantly greater proportions of participants had positive postpanel opinions than had negative postpanel opinions. Having consumers on review panels was reported to be beneficial (83.9% and 98.2% for scientists and consumers, respectively) and to not have drawbacks (74.7% and 87.3%, respectively). Our results support continued participation of consumers in our peer review process. The DOD program can serve as a model for other research programs considering consumer involvement.