Quantifying human performance for heterogeneous user populations using a structured expert elicitation

•Quantifying use-error risk can aid in maximizing system performance and safety.•Heterogeneous user populations present challenges for design validation studies.•An expert-driven methodology for quantifying human performance is proposed.•Method serves as a supplemental alternative to live participant studies.•The method is demonstrated on a diabetes population medical device use case study. Heterogeneous product user populations are common across many safety–critical domains. Catering to variable user needs is critical for designing safe and effective systems. Despite this, it is common for a 1-size-fits-all approach to be applied in design for these populations. Quantifying risk of use error throughout the design process can justify design decisions that maximize system performance and safety. Many regulatory agencies require consideration of user variability in design validation activities. However, there are practical challenges for integrating variable users into these activities. Adequately representing populations requires significant time and monetary commitments for subject recruitment. In addition, population access may be difficult in some cases. In this work, an alternative to traditional human factors design validation efforts is presented. Expert elicitation is proposed as a cost-effective means to quantify heterogenous user performance in the formative product design stages. The approach relies on the generation of generic physical and cognitive tasks that can be applied across use cases. The approach is demonstrated on the diabetes population, specially focusing on medical device use. The output of the demonstration are performance distributions for 27 task-user group pairs that can be integrated into design validation efforts to identify human error risks that require mitigation.