Multi‐scale statistical deformation based co‐registration of prostate MRI and post‐surgical whole mount histopathology

Background Accurate delineations of regions of interest (ROIs) on multi‐parametric magnetic resonance imaging (mpMRI) are crucial for development of automated, machine learning‐based prostate cancer (PCa) detection and segmentation models. However, manual ROI delineations are labor‐intensive and susceptible to inter‐reader variability. Histopathology images from radical prostatectomy (RP) represent the “gold standard” in terms of the delineation of disease extents, for example, PCa, prostatitis, and benign prostatic hyperplasia (BPH). Co‐registering digitized histopathology images onto pre‐operative mpMRI enables automated mapping of the ground truth disease extents onto mpMRI, thus enabling the development of machine learning tools for PCa detection and risk stratification. Still, MRI‐histopathology co‐registration is challenging due to various artifacts and large deformation between in vivo MRI and ex vivo whole‐mount histopathology images (WMHs). Furthermore, the artifacts on WMHs, such as tissue loss, may introduce unrealistic deformation during co‐registration. Purpose This study presents a new registration pipeline, MSERgSDM, a multi‐scale feature‐based registration (MSERg) with a statistical deformation (SDM) constraint, which aims to improve accuracy of MRI‐histopathology co‐registration. Methods In this study, we collected 85 pairs of MRI and WMHs from 48 patients across three cohorts. Cohort 1 (D1), comprised of a unique set of 3D printed mold data from six patients, facilitated the generation of ground truth deformations between ex vivo WMHs and in vivo MRI. The other two clinically acquired cohorts (D2 and D3) included 42 patients. Affine and nonrigid registrations were employed to minimize the deformation between ex vivo WMH and ex vivo T2‐weighted MRI (T2WI) in D1. Subsequently, ground truth deformation between in vivo T2WI and ex vivo WMH was approximated as the deformation between in vivo T2WI and ex vivo T2WI. In D2 and D3, the prostate anatomical annotations, for example, tumor and urethra, were made by a pathologist and a radiologist in collaboration. These annotations included ROI boundary contours and landmark points. Before applying the registration, manual corrections were made for flipping and rotation of WMHs. MSERgSDM comprises two main components: (1) multi‐scale representation construction, and (2) SDM construction. For the SDM construction, we collected N = 200 reasonable deformation fields generated using MSERg, verified throu