Attention-based CNN-BiLSTM for sleep state classification of spatiotemporal wide-field calcium imaging data

Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows for spatiotemporal recordings of neuronal activity in mice. When applied to the study of sleep, WFCI data are manually scored into the sleep states of wakefulness, non-REM (NREM) and REM by use of adjunct EEG and EMG recordings. However, this process is time-consuming, invasive and often suffers from low inter- and intra-rater reliability. Therefore, an automated sleep state classification method that operates on spatiotemporal WFCI data is desired. A hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to classify WFCI data into states of wakefulness, NREM and REM sleep. Sleep states were classified with an accuracy of 84 % and Cohen’s κ of 0.64. Gradient-weighted class activation maps revealed that the frontal region of the cortex carries more importance when classifying WFCI data into NREM sleep while posterior area contributes most to the identification of wakefulness. The attention scores indicated that the proposed network focuses on short- and long-range temporal dependency in a state-specific manner. On a held out, repeated 3-hour WFCI recording, the CNN-BiLSTM achieved a κ of 0.67, comparable to a κ of 0.65 corresponding to the human EEG/EMG-based scoring. The CNN-BiLSTM effectively classifies sleep states from spatiotemporal WFCI data and will enable broader application of WFCI in sleep research. •Manual scoring of WFCI based on EEG/EMG is time-consuming and invasive.•We proposed a hybrid method to classify sleep states based on WFCI data.•A CNN-BiLSTM network was employed to jointly learn spatiotemporal information.•The performance is comparable with human inter-rater performance based on EEG/EMG.•The spatial-temporal information of WFCI is pivotal for identifying sleep states.