Significant efficiency findings while controlling for the frequent confounders of CAI research in the PlanAlyzer project's computer-based, self-paced, case-based programs in anemia and chest pain diagnosis

Richard E. Clark in his widely published comprehensive studies and meta-analyses of the literature on computer assisted instruction (CAI) has decried the lack of carefully controlled research, challenging almost every study which shows the computer-based intervention to result in significant post-test proficiency gains over a non-computer-based intervention. We report on a randomized study in a medical school setting where the usual confounders found by Clark to plague most research, were carefully controlled. PlanAlyzer is a microcomputer-based, self-paced, case-based, event-driven system for medical education which was developed and used in carefully controlled trials in a second year medical school curriculum to test the hypothesis that students with access to the interactive programs could integrate their didactic knowledge more effectively and/or efficiently than with access only to traditional textual "nonintelligent" materials. PlanAlyzer presents cases, elicits and critiques a student's approach to the diagnosis of two common medical disorders: anemias and chest pain. PlanAlyzer uses text, hypertext, images and critiquing theory. Students were randomized, one half becoming the experimental group who received the interactive PlanAlyzer cases in anemia, the other half becoming the controls who received the exact same content material in a text format. Later in each year there was a crossover, the controls becoming the experimentals for a similar intervention with the cardiology PlanAlyzer cases. Preliminary results at the end of the first two full trials shows that the programs have achieved most of the proposed instructional objectives, plus some significant efficiency and economy gains. 96 faculty hours of classroom time were saved by using PlanAlyzer in their place, while maintaining high student achievement. In terms of student proficiency and efficiency, the 328 students in the trials over two years were able to accomplish the project's instructional objectives, and the experimentals accomplished this in 43% less time than the controls, achieving the same level of mastery. However, in spite of these significant efficiency findings, there have been no significant proficiency differences (as measured by current factual and higher order multiple choice post-tests) between the experimental and control groups. Very careful controls were used to avoid what Clark has found to be the most common confounders of CAI research. Accordingly, this research pr