Systemic lupus international collaborating clinics renal activity/response exercise: Comparison of agreement in rating renal response

Objective To assess the degree to which physicians agree with each other and with ratings obtained with 3 existing responder indices, in rating the response to treatment of lupus nephritis. Methods Lupus nephritis patient medical records from 125 pairs of visits (6 months apart) were used to create renal response scenarios. Seven nephrologists and 22 rheumatologists rated each scenario as demonstrating complete response, partial response, same, or worsening. The plurality (most frequent) rating of renal response by the physicians was compared with the calculated score from the renal component of the British Isles Lupus Assessment Group (BILAG) index (original and updated [2004] version) and of the Responder Index for Lupus Erythematosus (RIFLE). The degree of agreement among the physicians was assessed by calculating intraclass correlation coefficients (ICCs). The degree of agreement between the plurality physician rating and ratings obtained with the established response indices was assessed using the kappa statistic. Results The ICC among all physicians was 0.64 (0.62 for nephrologists and 0.67 for rheumatologists). The chance‐adjusted measure of agreement (kappa coefficient) between the plurality physician rating and the calculated score obtained using established indexes was 0.50 (95% confidence interval [95% CI] 0.38–0.61) for the RIFLE, 0.14 (95% CI 0.03–0.25) for the original BILAG, and 0.23 (95% CI 0.21–0.44) for the BILAG 2004. Conclusion These findings indicate that rheumatologists as a group and nephrologists as a group have equal agreement in their rating of renal response. There was moderate agreement between plurality physician ratings and ratings obtained using the renal component of the RIFLE. Ratings of response using an index based on the original BILAG did not have good agreement with the plurality physician rating.