Harnessing multiple models for outbreak management

Expert elicitation methods and a structured decision-making framework will help account for risk and uncertainty The coronavirus disease 2019 (COVID-19) pandemic has triggered efforts by multiple modeling groups to forecast disease trajectory, assess interventions, and improve understanding of the pathogen. Such models can often differ substantially in their projections and recommendations, reflecting different policy assumptions and objectives, as well as scientific, logistical, and other uncertainty about biological and management processes ( 1 ). Disparate predictions during any outbreak can hinder intervention planning and response by policy-makers ( 2 , 3 ), who may instead choose to rely on single trusted sources of advice, or on consensus where it appears. Thus, valuable insights and information from other models may be overlooked, limiting the opportunity for decision-makers to account for risk and uncertainty and resulting in more lives lost or resources used than necessary. We advocate a more systematic approach, by merging two well-established research fields. The first element involves formal expert elicitation methods applied to multiple models to deliberately generate, retain, and synthesize valuable individual model ideas and share important insights during group discussions, while minimizing various cognitive biases. The second element uses a decision-theoretic framework to capture and account for within- and between-model uncertainty as we evaluate actions in a timely manner to achieve management objectives.