Augmentation of the think aloud method with users’ perspectives for the selection of a picture archiving and communication system

[Display omitted] •This study augmented Think Aloud (TA) with users’ perspective for PACS selection.•The results highlighted the effect of different usability characteristics of PACS user interfaces on the selection of a PACS.•Involvement of different groups of users in the evaluation results in identifying more specific usability problems.•The augmented method identified specific new usability problems in PACSs. Users attitude toward a picture archiving and communication system (PACS) and their interaction with this system are among the most important factors that influence its acceptance. This study aimed to augment the user's interaction with the user's perspective to select a usable PACS among three systems available on the market. We augmented the think aloud (TA) usability evaluation method with the Post-Study System Usability Questionnaire (PSSUQ) to compare user interaction problems of three PACS user interfaces. Four radiologists and four internist physicians participated in this study. Usability characteristics including efficiency, effectiveness, learnability, error, and satisfaction were used to assess the usability of each PACS. There was a significant difference in efficiency (p = 0.01), effectiveness (p = 0.005), learnability (p = 0.001), and satisfaction (p = 0.009). However, no significant difference in the number of errors (p = 0.18), mouse clicks and keystrokes (p = 0.12), and the number of usability problems (p = 0.6) were observed among the three PACS systems studied. This study showed that applying the proposed approach to augment TA with the user's perspective addresses almost all of the theoretical aspects of usability and can be employed to select the most usable PACS.