Exploring the Usability of Mobile Apps Supporting Radiologists’ Training in Diagnostic Decision Making

Abstract Purpose The objective of this study was to conduct a usability evaluation of mobile apps for supporting education and training in radiologic diagnostic decision-making processes. Methods Of 381 mobile apps available at two major stores (Google Play and iTunes), eight iOS apps were selected for laboratory-based usability tests. Six staff radiologists completed eight app-specific task sets, using a think-aloud strategy. The triangular methods approach included quantitative performance measures, System Usability Scale (SUS), and qualitative thematic analysis using heuristic usability principles of usability issues. Results Overall, radiologists achieved higher than 70% success, with favorable SUS scores, in completing the tasks for seven of the eight apps. However, task success rate and SUS score had a weak relation ( r = 0.23), indicating that the perceived usability may not reflect the holistic usability of the app. Task analysis and self-report revealed 108 usability issues, which were condensed to 55 unique issues and categorized by nine usability themes and mapped to ten usability heuristics. Nonintuitive functionality (eg, nonintuitive or misleading labels) was the most frequent theme observed, leading to inefficient navigation. These usability findings were consistent with the 13 improvements the radiologists suggested. Conclusions This study demonstrates the feasibility of usability evaluation of radiology mobile apps and suggests potential improvements in the development of radiology mobile apps. This study also suggests that proficiency with mobile devices may not be equivalent to being an expert user, proficient in using the apps.