A head mounted augmented reality design practice for maintenance assembly: Toward meeting perceptual and cognitive needs of AR users

Head Mounted Display (HMD) based Augmented Reality (AR) is being increasingly used in manufacturing and maintenance. However, limited research has been done to understand user interaction with AR interfaces, which may lead to poor usability, risk of occupational hazards, and low acceptance of AR systems. This paper uses a theoretically-driven approach to interaction design to investigate the impact of different AR modalities in terms of information mode (i.e. video vs. 3D animation) and interaction modality (i.e. hand-gesture vs. voice command) on user performance, workload, eye gaze behaviours, and usability during a maintenance assembly task. The results show that different information modes have distinct impacts compared to paper-based maintenance, in particular, 3D animation led to a 14% improvement over the video instructions in task completion time. Moreover, insights from eye gaze behaviours such as number of fixations and transition between Areas of Interest (AOIs) revealed the differences in attention switching and task comprehension difficulty with the choice of AR modalities. While, subjective user perceptions highlight some ergonomic issues such as misguidance and overreliance, which must be considered and addressed from the joint cognitive systems’ (JCSs) perspective and in line with the predictions derived from the Multiple Resources Model. •Multiple Resource Model (MRM) and the Joint Cognitive Systems’ (JCSs) perspective were applied to examine HMD AR modalities.•3D animation led to a 14% improvement over the video instructions in task completion time.•Task completion time was positively correlated with an increase in the number of fixations.•The ergonomic issues of HMD AR modalities can be addressed from JCS’s perspective and in line with MRM predictions.