Knowledge Acquisition Evaluation Using Simulated Experts

Evaluation of knowledge acquisition (KA) is difficult in general because of the costs of using a human expert. In this paper, we use a general simulation framework to evaluate some aspects of KA. We focus on the importance of acquiring domain ontological structures and the use of stored or cornerstone cases to validate changes. We find that the for a higher level of expertise, an ontology is very useful, but cornerstone cases less so, but the weaker the level of expertise, the more valuable the cornerstone cases and the less helpful an ontology.