A Multi-Modal Approach for Crop Health Mapping Using Low Altitude Remote Sensing, Internet of Things (IoT) and Machine Learning

The agriculture sector holds paramount importance in Pakistan due to the intrinsic agrarian nature of the economy. Pakistan has its GDP based on agriculture, however it relies on manual monitoring of crops, which is a labour intensive and ineffective method. In contrast to this, several cutting edge technology-based solutions are being employed in the developed countries to enhance the crop yield with the optimal use of resources. To this end, we have proposed an integrated approach for monitoring crop health using IoT, machine learning and drone technology. The integration of these sensing modalities generate heterogeneous data which not only varies in nature (i.e. observed parameter) but also has different temporal fidelity. The spatial resolution of these methods is also different, hence, the optimal integration of these sensing modalities and their implementation in practice are addressed in the proposed system. In our proposed solution, the IoT sensors provide the real-time status of environmental parameters impacting the crop, and the drone platform provide the multispectral data used for generating Vegetation Indices (VIs) such as Normalized Difference vegetation Index (NDVI) for analyzing the crop health. The NDVI provides information about the crop based on the chlorophyll content, which offers limited information regarding the crop health. In order to obtain a rich and detailed knowledge about crop health, the variable length time series data of IoT sensors and multispectral images were converted to a fixed-sized representation to generate crop health maps. A number of machine and deep learning algorithms were applied on the collected data wherein deep neural network with two hidden layers was found to be the most optimal model among all the selected models, providing an accuracy of (98.4%). Further, the health maps were validated through ground surveys and by agriculture experts due to the absence of reference data. The proposed research is basically an indigenous, technology based agriculture solution capable of providing important insights into the crop health by extracting complementary features from multi-modal data set, and minimizing the crop ground survey effort, particularly useful when the agriculture land is large in size.