An Ensemble Deep Learning Approach Combining Phenotypic Data and fMRI for ADHD Diagnosis

As a common neurological disorder in early childhood and adolescence, an efficient and accurate diagnosis of Attention-Defect/Hyperactivity Disorder (ADHD) has always been one of the important goals in the field of psychiatry. However, most of the current diagnostic methods are based on a single table or fMRI images, which may result in the loss of much complementary information useful for diagnosis. Therefore, this paper presents a strategy for multimodal data fusion and an ensemble learning model (Trans3D-ensemble) to classify the ADHDs and typicals. The base classifiers of Trans3D-ensemble include two parts, Trans3D (Transformer for 3D images) to extract spatio-temporal features from fMRI images and random forest to extract clinical features from phenotypic data. Specifically, Trans3D utilizes 3D-CNN to capture volumetric spatial information and convert the 3D images into patch embeddings. Meanwhile, temporal pooling operation fuses the images tokens output by Transformer encoders across time and obtains the representative features of fMRI samples. Existing methods basically only build the above independent models, and few consider the effective combination of the those for diagnosis and treatment. Looking into this issue, we use stacking, one of ensemble learning methods, on multimodal data by merging the outputs from the two base classifiers. The proposed method reaches excellent results compared to most methods based on single modality and gets the accuracy of 74.5% on ADHD-200 data set, demonstrating the potential of the combination of phenotypic data and fMRI for ADHD diagnosis. Our strategy of multimodal data fusion provides a novel comprehensive diagnosis mode and the above results show that the proposed Trans3D-ensemble can effectively improve the auxiliary diagnosis of ADHD.