MULTISTREAM FUSION ENCODER FOR PROSTATE LESION SEGMENTATION AND CLASSIFICATION

The present invention provides a flexible, light-weighted and efficient multistream fusion encoder which can be easily integrated into multistream convolutional neural networks to perform segmentation and classification tasks on MRI images registered with different modalities. The encoder allows fusion of extracted feature maps in multiple streams on a layer-by-layer basis and generates the output of each stream by adding the corresponding convolutional output with an adaptively weighted fusion map computed from outputs of all streams. Adaptive weighting of fusion maps at each layer allows flexibility in highlighting different image modalities according to their relative influence on the segmentation/classification performance. The fusion encoder can also play an important role in the segmentation-classification workflow in biopsy and focal therapy planning.