An experimental sky-image-derived cloud validation dataset for Sentinel-2 and Landsat 8 satellites over NASA GSFC

[Display omitted] •A new cloud validation dataset GSFC-Cloud is introduced.•GSFC-Cloud is based on ground-based images of the sky.•GSFC-Cloud available for Landsat 8 and Sentinel-2.•Improvements are proposed for LaSRC Sentinel-2 cloud detection algorithm.•Parallax feature for LaSRC Sentinel-2 implemented with a phase correlation approach. Availability of a reliable cloud mask for optical satellite imagery is a prerequisite, when generating high-quality high-level geoinformation products. Creation of a reference (ground truth) cloud mask for moderate spatial resolution sensors, such as Operational Land Imager (OLI) aboard Landsat 8 and Multispectral Instrument (MSI) aboard Sentinel-2A/B satellites, is a challenging and time-consuming task. Existing reference datasets were mainly produced through photointerpretation of satellite images by an analyst, which can introduce subjectivity in detecting clouds. Therefore, other methods for generating cloud reference data shall be explored and evaluated that can complement existing datasets. In this paper, we document generation and provide the description of a new reference cloud dataset, named GSFC-Cloud, which is based on the extensive use of ground-based images of the sky. The dataset is collected over the same area, covers various cloud conditions, and is available for six Landsat 8 and twenty-eight Sentinel-2 scenes spanning the period of September 2017 to November 2018. The dataset is available in the vector format, so cloud masks at various spatial resolutions can be validated. We also describe a system to automate the process of ground-based data collection using low-cost off-the-shelf parts with the long-term objective to replicate this set-up in multiple locations around the world. We use the proposed dataset to validate and improve the Land Surface Reflectance Code (LaSRC) for cloud detection in Sentinel-2 imagery. We show that adding a parallax feature to estimate a subpixel shift between red and green bands with a phase correlation method can reduce overdetection of clouds and improve performance of LaSRC.