A joint learning Im-BiLSTM model for incomplete time-series Sentinel-2A data imputation and crop classification

•Reduced the impact of cloud contamination on multi-temporal crop classification.•Im-BiLSTM model is proposed for crop classification of incomplete time-series data.•Im-BiLSTM model jointly performs missing data imputation and crop classification.•Im-BiLSTM model classification outperforms BiLSTM model.•Improved the deep learning Im-BiLSTM model interpretability. Multi-temporal deep learning approaches can make full use of crop growth patterns and phenological characteristics, resulting in excellent crop classification performance in large areas. However, obtaining complete time-series remote sensing images during the growing season is challenging due to cloud contamination. Hence, given time-series multispectral data, it is important to impute missing data and accurately classify crops. A novel Imputation-BiLSTM model (Im-BiLSTM) was developed based on Bidirectional Long Short-term Memory network (BiLSTM) to jointly perform missing data imputation and crop classification. The Im-BiLSTM model regards missing data as network variables, which are efficiently updated during backpropagation. Im-BiLSTM treats the interaction between imputation and classification tasks, reducing the error and uncertainty caused by the separation operation of imputation to classification. Furthermore, we improved the interpretability of the Im-BiLSTM model by evaluating the importance of input features and visualizing hidden state units. In Shawan County, Xinjiang, China, we acquired a total of 10 Sentinel-2A images from April to October 2016, of which 3 images lost partial data due to cloud cover. The Im-BiLSTM model was applied to incomplete time-series data containing 10 time-steps for pixel-level crop classification, and the BiLSTM model was constructed based on cloud-free images for comparison. The performance of the proposed model was tested in four different cases of images missing and missing rates. The results showed that the classification of the Im-BiLSTM model outperformed the BiLSTM model, the overall accuracy was improved by a maximum of 4.2%, and the F1-scores of spring corn and tomato was improved by a maximum of 16.1% and 21.4%, respectively. Therefore, the Im-BiLSTM model can effectively improve classification performance by jointly imputing missing data. The imputation results (the coefficient of determination values range 0.4 ∼ 0.9) indicated that the bands with the larger contribution to the classification had higher imputation accuracy. Feature importance ev