Results of a Multi-Institution Deformable Registration Accuracy Study (MIDRAS)

Purpose To assess the accuracy, reproducibility, and computational performance of deformable image registration algorithms under development at multiple institutions on common datasets. Methods and Materials Datasets from a lung patient (four-dimensional computed tomography [4D-CT]), a liver patient (4D-CT and magnetic resonance imaging [MRI] at exhale), and a prostate patient (repeat MRI) were obtained. Radiation oncologists localized anatomic structures for accuracy assessment. Algorithm accuracy was determined by comparing the computer-predicted displacement at each bifurcation point with the displacement computed from the oncologists' annotations. Thirty-seven academic institutions and medical device manufacturers with published evidence of active deformable image registration capabilities were invited to participate. Results Twenty-seven groups agreed to participate; 6 did not return results. Sixteen completed the liver 4D-CT, 12 the lung 4D-CT, 3 the prostate MRI, and 3 the liver MRI-CT. The range of average absolute error for the lung 4D-CT was 0.6–1.2 mm (left–right [LR]), 0.5–1.8 mm (anterior–posterior [AP]), and 0.7–2.0 mm (superior–inferior [SI]); the liver 4D-CT was 0.8–1.5 mm (LR), 1.0–5.2 mm (AP), and 1.0–5.9 mm (SI); the liver MRI-CT was 1.1–2.6 mm (LR), 2.0–5.0 mm (AP), and 2.2–2.6 mm (SI); and the repeat prostate MRI prostate datasets was 0.5–6.2 mm (LR), 3.1–3.7 mm (AP), and 0.4–2.0 mm (SI). Conclusions An infrastructure was developed to assess multi-institution deformable registration accuracy. The results indicate large discrepancies in reported shifts, although the majority of deformable registration algorithms performed at an accuracy equivalent to the voxel size, promising to improve treatment planning, delivery, and assessment.