Adapted Bland–Altman method was used to compare measurement methods with unequal observations per case

Abstract Objectives To describe an adjustment of the Bland–Altman approach to evaluate possible patterns of discord between two measurement methods with an unequal number of observations per case. Study Design and Setting Two methods of adaptation were compared using self-assessed general well-being scores (one individual rater) and scores given by multiple external raters for illustration. Both empirical data derived from the Older Person's Relevant Outcome of Care Score study and simulated data were used. Results When the mean of a number of assessments [ X ¯ ] is compared with a single assessment [ Y ], the variation [ X ¯ − Y ] (vertical axis) will be correlated with the mean of X ¯ and Y (horizontal axis) because the means of scores given by a larger group of raters tend to be less extreme than the scores given by individual raters. In contrast, in the absence of discord patterns, the variation [ X ¯ − Y ] will not be correlated with the mean of all observations X i and Y j made irrespective of who made them as denominator in the Bland–Altman plot. Thus, the second approach should be used for the valuation. Conclusion The Bland–Altman approach needs correct adaptations, else artificial biases may occur.