CloudS2Mask: A novel deep learning approach for improved cloud and cloud shadow masking in Sentinel-2 imagery

The Sentinel-2 satellite constellation produces high-resolution multispectral data, covering the entire Earth's land surface every five days. However, the use of this data is significantly impacted by the presence of pixels affected by cloud and shadow. Precise and efficient cloud and cloud shadow masking methods are required for the automated use of this data. Here, we present CloudS2Mask, a new and open-source Python deep-learning library to detect cloud and cloud shadows in Sentinel-2 data accurately and efficiently. CloudS2Mask is trained on multiple large global datasets including CloudSEN12, KappaSet, Sentinel-2 Cloud Mask Catalogue, Sentinel-2 reference cloud masks generated by an active learning method and a custom CloudS2Mask dataset. Using techniques such as progressive resizing, hard fine-tuning, and transfer learning, CloudS2Mask outperforms published models when evaluated on both the CloudSEN12 and the PixBox datasets in terms of classification accuracy. Compared to the next best-published results, CloudS2Mask improves Balanced Overall Accuracy metrics on the CloudSEN12 dataset by 1.2% for cloud, 2.8% for cloud shadow and 1.1% for clear classification. CloudS2Mask provides a range of customisable options combined with novel pre-processing and post-processing approaches to meet task-specific needs, enabling mean scene processing speeds up to 44 times faster than current state-of-the-art methods. The application of novel techniques in CloudS2Mask, combined with its accuracy and processing speed, underscores its potential to streamline the automated cloud and cloud shadow detection of Sentinel-2 data. [Display omitted] •A novel deep learning segmentation approach for Sentinel-2 cloud and cloud shadow.•CloudS2Mask achieves the highest accuracy measured on CloudSEN12 and PixBox datasets.•High accuracy over various surface types and through time.•Rapid scene processing, as low as 2.2 s per scene.