Dual-channel neural network for instance segmentation of synapse

Detection and segmentation of neural synapses in electron microscopy images are the committed steps for analyzing neural ultrastructure. To date, manual annotation of the structure in synapses has been the primary method, which is time-consuming and restricts the throughput of data acquisition. Recent studies have utilized a series of deformations based on a segmentation model for the detection and segmentation of transmission electron microscope images. However, the analysis of synaptic segmentation and statistics still lacks sufficient automation and high-throughput. Therefore, we developed a dual-channel neural network instance segmentation model with weighted top-down and multi-scale bottom-up schemes, which aid in accurately detecting and segmenting synaptic vesicles and their active zones within presynaptic membranes in complex environments. In addition, we proposed a masked self-supervised pre-training model based on the latest convolutional architecture to improve performance in downstream segmentation tasks. By comparing our model to other state-of-the-art methods, we determined its viability and accuracy. The applicability of our model is thoroughly demonstrated by distinct application scenarios for neurobiological research. These findings indicate that the dual-channel neural network could facilitate the analysis of synaptic structures for the advancement of biomedical research and electron microscope reconstruction techniques. •A novel dual-channel neural network (DCNN) is proposed to improve the recognition, segmentation, and quantification of complex neurobiological images.•To segment and classify neural synapse instances in transmission electron microscope (TEM) images, the top-down channel with weight regression loss and the bottom-up channel with multi-scale segmentation loss were utilized.•Experimental study shows the efficiency of a ConvNeXt-based masked self-supervised pre-training model for downstream segmentation tasks.•The DCNN outperforms other advanced models while it applies to analyzing the neurobiological images.