Interrater reliabilities for two measures of nursing care quality

The intraclass correlation coefficient (ICC) was used to estimate the interrater reliability for two instruments designed to measure the quality of nursing care. The two instruments tested were the Quality Patient Care Scale (QualPacs) and the Methodology for Monitoring Quality of Nursing Care (Rush‐Medicus instrument). The following minimal standards were proposed for evaluating the results of interrater reliability: (a) the ICC estimate obtained must have a value of at least .75, and (b) the 95% confidence interval should not include zero, in order to ensure that the obtained value differs from zero in a nonchance fashion. The results demonstrated that the QualPacs instrument did not meet these criteria in any instance for the six subscales and one total‐scale score for two sets of observers in two replications. The Rush‐Medicus instrument met the criteria in only 12 of 20 instances of testing (five of the six subscales for two sets of observers in two replications). The authors highly recommend the routine performance of interrater reliability testing using the ICC prior to and during data collection and acceptance of the minimal standards of empirical verification suggested previously.