Need for multi-aspect measures to support evaluation of complex human-computer interfaces

Inherent to the objective evaluation of a system is the use of evaluative measures. These are metrics or formulas designed to represent one or more aspects of system performance. As the system becomes more complex, it stands to reason that the measures needed to correctly reveal its performance may have to become correspondingly more complex as well. This is particularly evident when the HCl system evaluation of interest must encompass each of: (1) the system, (2) its human user and (3) the dynamic interaction between them. The paper portrays the types of measures that have been and continue to be used. Then, it characterizes the design, testing, and validation of more complex measures for use in evaluation methodology applicable to complex systems. A combination of three major evaluative aspects-synergism, efficiency, and effectiveness (SEE)-is shown to be useful in guiding the design of such measures.