Rheumatoid arthritis antirheumatic drug trials: effects of a standardized instructional videotape on the reliability of observer-dependent outcome measures

A study was designed to assess the effects of a standardized instructional videotape on reducing interobserver variability for several commonly used observer-dependent outcome measures. During a single day, six rheumatologists independently examined six patients with rheumatoid arthritis (RA) in a predetermined order using a Latin square design, before and after viewing a standardized videotape demonstrating 13 examination techniques. Reliability coefficients were calculated based on the variance components of the analysis of variance (ANOVA) table. Prestandardization reliability coefficients were >0.80 for all measures and remained above 0.80 following standardization. It is usually assumed that serial measurement in clinical trials should be performed by the same assessor because of concern regarding interobserver variability. However, the high levels of prestandardization interobserver reliability observed in this study indicate that, for these variables, serial measurements in a clinical trial could be made by different assessors, assuming they were equally skilled. This observation has important implications for outcome measurement in RA clinical trials. Although high levels of prestandardization reliability precluded the demonstration of any significant effect, we speculate that the videotape might be effective in training less-experienced assessors. Reductions in observer variability have the potential to diminish sample size requirements for RA antirheumatic drug studies. The use of a videotape to achieve this goal offers cost and convenience advantages over one-on-one training procedures, and this method should be further assessed in less-experienced assessors.