Web evaluation: Heuristic evaluation vs. user testing

It is very important that designers recognize the benefits and limitations of different usability inspection methods. This is because the quality of the usability evaluation is dependent on the method used. Two of the most popular usability evaluation techniques are user testing and heuristic analysis. The main objective of this study was to compare the efficiency and effectiveness between user testing and heuristic analysis in evaluating four different commercial web sites. The results showed that both user testing and heuristic analysis addressed different usability problems. Analysis by severity of problems found and diminishing return analysis model on the relationship between the number of new problems discovered with users and evaluators used showed that both methods are equally efficient and effective in addressing different categories of usability problems. These significant differences found between these two methods suggested that the two methods are complimentary and should not be competing. In order for better evaluation results, both user testing and heuristic analysis are still needed. The research findings from this study will be of particular value to the web development industry and communities. Knowledge regarding the differences between user testing and heuristic evaluation will enable appropriate business decisions to be made on when and how to apply these methods to improve the overall efficiency of the design process.