Utilizing Experts' Judgment in the Process of Curriculum Evaluation

The employment of judgment as an evaluation tool at the early stage of curriculum development is advantageous. The reasons why this is so are: (1) writers exhibit more readiness to change elements of the program during the early stages of curriculum development; (2) there is great economic advantage in using such an evaluation tool; and (3) utilization of expert judgment may decrease the time needed for program development. The limitations to the utilization of judgmental data are related to what to judge, specifications and outcomes, specifications and planned activities, planned and actual activities, and actual activities and outcomes. The present paper does not contain a detailed list of questions that may be presented to experts, but rather concentrates on indicating several aspects of the program that can be judged. Four major aspects of the fit of planned activities to the program specifications are content, presentation, learning apparatus, and relatedness to stated objectives. Prediction of future events relate to actual activities, student interest, difficulty level, teachers' reaction, and community reaction. The selection of experts for judging the fit between the program specifications and the planned learning activities should be made on the basis of logical considerations. Three methods of examining the validity of judgmental data are: (1) to quantify the degree of consensus obtained among experts with regard to different issues; (2) the employment of the "shuffle test"; and (3) by experimental validation. (DB)