Evaluation of clinical practice guideline quality: comparison of two appraisal tools

Abstract Objective The tools used for critically appraising the quality of clinical practice guidelines are complex and not suitable for the busy end users. So rapid, effective and simple instruments are more preferred. The aim of this study is to compare two critical appraisal tools: iCAHE as a rapid instrument and AGREE II as a complex instrument on guideline quality assessment. Material and Methods The diabetes mellitus guidelines of the Scottish Intercollegiate Guidelines Network (SIGN), the National Institute for Health and Clinical Excellence (NICE), the International Diabetes Federation (IDF) and the Society of Endocrinology and Metabolism of Turkey (SEMT) were assessed separately by four appraisers using the iCAHE and AGREE II instruments. The mean iCAHE criteria scores and the total and domain AGREE II scores given by the four appraisers are presented for each guideline. Results No statistically significant difference was detected between the iCAHE scale scores of the guidelines evaluated (P = 0.063). The rank of the guidelines according to their average total iCAHE and AGREE II instrument scores was similar. The iCAHE mean scores of the guidelines were as follows: NICE, 92.85%; SIGN, 92.85%; IDF, 66.07% and SEMT, 73.21%. The AGREE II mean scores of the guidelines were as follows: NICE, 87.13%; SIGN, 78.25%; IDF, 53.44% and SEMT, 53.22%. Conclusions In addition to being a quality scale, the iCAHE checklist is easy, practical and short to implement. It also helps the users to understand the quality of the guideline in a shorter time. To increase the use of guidelines, it is important that users with little experience and time use the iCAHE scale as a rapid appraisal tool, but more studies are needed to decide the best appraisal tool.