A residual graph convolutional network with spatio-temporal features for autism classification from fMRI brain images

Diagnosing autism spectrum disorder (ASD) remains a challenge due to its complexity and insufficient evidence available for its diagnosis. Recent research in the field of psychiatry suggests that there are no clear causes for ASD, but a hypothesis is raised that abnormalities in the superior temporal sulcus (STS), which is connected to visual cortex regions, can serve as a critical indicator of ASD. Inspired by this hypothesis, this paper proposes a deep learning model with two parts to diagnose ASD by utilizing functional brain connectivity between STS and visual cortex. The first part is a residual attention network that selectively extracts the structural and temporal features from 4D brain images while maintaining dynamic connectivity between the two regions. The second part is a graph convolutional network that determines ASD from the graph with 39 nodes constructed by the residual attention network. Experiments with the fMRI data from 800 patients known as ABIDE (Autism Brain Imaging Data Exchange) and 10-fold cross-validation show that the proposed model outperforms the state-of-the-art methods by achieving 11.37%p improvement in the ASD classification. Additional analyses justify each part of the proposed model through an ablation study and various visualizations. •We propose a novel deep learning model for diagnosing autism spectrum disorder.•It extracts spatial–temporal features of fMRI brain images with residual attention network.•It classifies the features for ASD diagnosis with graph convolutional network with attention.•Experiments with large benchmark data shows the model outperforms the SOTA models.•10-fold CV and various analyses verify the usefulness of the proposed model.