Transfer learning on fused multiparametric MR images for classifying histopathological subtypes of rhabdomyosarcoma

•A deep-learning based CADx for diagnosis of embryonal and alveolar subtypes.•Diagnosis has been performed by solely by analyzing multiparametric MR images.•Created a fusion of diffusion-weighted and T1-weighted MR scans.•A pre-trained deep neural network has been fine-tuned based on the fused images.•Achieved 85% cross validation accuracy for classifying the two RMS subtypes.•The system can provide an efficient and reproducible diagnosis with less human interaction. This paper presents a deep-learning-based CADx for the differential diagnosis of embryonal (ERMS) and alveolar (ARMS) subtypes of rhabdomysarcoma (RMS) solely by analyzing multiparametric MR images. We formulated an automated pipeline that creates a comprehensive representation of tumor by performing a fusion of diffusion-weighted MR scans (DWI) and gadolinium chelate-enhanced T1−weighted MR scans (MRI). Finally, we adapted transfer learning approach where a pre-trained deep convolutional neural network has been fine-tuned based on the fused images for performing classification of the two RMS subtypes. We achieved 85% cross validation prediction accuracy from the fine-tuned deep CNN model. Our system can be exploited to provide a fast, efficient and reproducible diagnosis of RMS subtypes with less human interaction. The framework offers an efficient integration between advanced image processing methods and cutting-edge deep learning techniques which can be extended to deal with other clinical domains that involve multimodal imaging for disease diagnosis.