Implementation of a situation awareness assessment tool for evaluation of human-robot interfaces

In this paper, we outline a methodology for evaluating the situation awareness (SA) provided by a supervisory interface for an autonomous on-road vehicle. Our goal is to be able to use the evaluations to compare interface designs with respect to how well each facilitates the users' acquisition of SA. We used Endsely's Situation Awareness Global Assessment Technique (SAGAT) (Endsely, 1988) and developed scenarios and assessment questions appropriate for supervisors of autonomous on-road driving vehicles. We describe the results of two experiments used to refine our SA assessment implementation. In a third experiment, we applied the refined implementation to a graphical user interface we developed to test the sensitivity of our SAGAT implementation. We discuss the results of this experiment and implications for applying the SAGAT methodology to supervisory user interfaces for autonomous vehicles.