Segmentation and quantification of infarction without contrast agents via spatiotemporal generative adversarial learning

•Simultaneous segmentation and quantification of infarction without contrast agent.•Learning time-series images by using spatiotemporal pyramid representation.•Improving performance by exploiting the commonalities and differences across tasks.•Embedding segmentation and quantification tasks into adversarial learning. [Display omitted] Accurate and simultaneous segmentation and full quantification (all indices are required in a clinical assessment) of the myocardial infarction (MI) area are crucial for early diagnosis and surgical planning. Current clinical methods remain subject to potential high-risk, nonreproducibility and time-consumption issues. In this study, a deep spatiotemporal adversarial network (DSTGAN) is proposed as a contrast-free, stable and automatic clinical tool to simultaneously segment and quantify MIs directly from the cine MR image. The DSTGAN is implemented using a conditional generative model, which conditions the distributions of the objective cine MR image to directly optimize the generalized error of the mapping between the input and the output. The method consists of the following: (1) A multi-level and multi-scale spatiotemporal variation encoder learns a coarse to fine hierarchical feature to effectively encode the MI-specific morphological and kinematic abnormality structures, which vary for different spatial locations and time periods. (2) The top-down and cross-task generators learn the shared representations between segmentation and quantification to use the commonalities and differences between the two related tasks and enhance the generator preference. (3) Three inter-/intra-tasks to label the relatedness discriminators are iteratively imposed on the encoder and generator to detect and correct the inconsistencies in the label relatedness between and within tasks via adversarial learning. Our proposed method yields a pixel classification accuracy of 96.98%, and the mean absolute error of the MI centroid is 0.96 mm from 165 clinical subjects. These results indicate the potential of our proposed method in aiding standardized MI assessments.