Evaluating expert systems: how to ensure an independent assessment

Evaluation of an expert system provides unique challenges in an organizational setting. Often the proposed system is provided by an independent supplier who is not in a position to provide an unbiased assessment of the effectiveness or merit of the system. Even when the system is developed by personnel within the same firm or by the users themselves, these individuals, being instrumental in the design and development of the system, lack the autonomy necessary to evaluate the system. This article describes one evaluation process which was applied by the authors. It is an adaptable approach which provides the needed independent assessment without being excessively cumbersome or difficult to administer.