Intra‐ and Inter‐Rater Agreement Describing Myometrial Lesions Using Morphologic Uterus Sonographic Assessment: A Pilot Study

Objectives To evaluate the intra‐ and inter‐rater agreement for myometrial lesions using Morphologic Uterus Sonographic Assessment terminology. Methods Thirteen raters with high (n = 6) or medium experience (n = 7) assessed 30 3‐dimensional ultrasound clips with (n = 20) and without (n = 10) benign myometrial lesions. Myometrial lesions were reported as poorly or well defined and then systematically evaluated for the presence of individual features. The clips were blindly assessed twice (at a 2‐month interval). Intra‐ and inter‐rater agreements were calculated with κ statistics. Results The reporting of poorly defined lesions reached moderate intra‐rater agreement (κ = 0.49 [high experience] and 0.47 [medium experience]) and poor inter‐rater agreement (κ = 0.39 [high experience] and 0.25 [medium experience]). The reporting of well‐defined lesions reached good to very good intra‐rater agreement (κ = 0.73 [high experience] and 0.82 [medium experience]) and good inter‐rater agreement (κ = 0.75 [high experience] and 0.63 [medium experience]). Most individual features associated with ill‐defined lesions reached moderate intra‐ and inter‐rater agreement among highly experienced raters (κ = 0.41–0.60). The least reproducible features were myometrial cysts, hyperechoic islands, subendometrial lines and buds, and translesional flow (κ = 0.11–0.34). Most individual features associated with well‐defined lesions reached moderate to good intra‐ and inter‐rater agreement among all observers (κ = 0.41–0.80). The least reproducible features were a serosal contour, asymmetry, a hyperechoic rim, and fan‐shaped shadows (κ = 0.00–0.35). Conclusions The reporting of well‐defined lesions showed excellent agreement, whereas the agreement for poorly defined lesions was low, even among highly experienced raters. The agreement on identifying individual features varied, especially for features associated with ill‐defined lesions. Guidelines on minimum requirements for features associated with ill‐defined lesions to be interpreted as poorly defined lesions may improve agreement.