Deriving knowledge representation guidelines by analyzing knowledge engineer behavior

Knowledge engineering research has focused on proposing knowledge acquisition techniques, developing and evaluating knowledge representation schemes and engineering tools, and testing and debugging knowledge-based systems. Few formal studies have been conducted on understanding the behaviors and roles of knowledge engineers. Applying the theory of mental models, this paper describes a think aloud verbal protocol study to determine an empirical basis for understanding: (1) how knowledge engineers extract domain knowledge from textual sources; and (2) the cognitive mechanisms by which they engage various knowledge representation schemes to represent that knowledge acquired. The results suggest that knowledge representation is not simply a translation of acquired knowledge to a knowledge representation. Instead, it is an iterative process of selective querying of acquired knowledge, and continuous refinement of a model leveraging, not only on acquired knowledge from domain experts, but also from the knowledge engineer. From the findings of empirical studies, a set of guidelines is derived to support the training and development of better knowledge representation schemes, representation processes, and knowledge engineering tools. ► We use protocol analysis to explore how knowledge engineers think. ► We find knowledge engineers perform 6 kinds of cognitive actions. ► We develop a flowchart illustrating the cognitive flow of knowledge engineers.