An analysis of guidelines for expert reviewers

Forty eight sets of recommendations identified by their authors as practical heuristics for the evaluation and revision of instructional materials were reviewed and consolidated. These guidelines were extracted from professional journals, book chapters and independent publications. Initially, all items were compiled, and then sorted into three categories according to their specific focus: content (subject matter), design, and presentation. In a second sort, identical items were eliminated and semantically equivalent items were grouped together. Three independent judges performed the same tasks for reliability. The outcome is a comprehensive list of 67 items, representing all of the reviewed guidelines. This instrument could be a successful aid in identifying deficiencies of instructional materials in the areas of content, design and presentation.