Single-plant broccoli growth monitoring using deep learning with UAV imagery

•Integrating UAV imagery with object detection and semantic segmentation.•Generating a visualized growth map on a single-plant basis using deep learning.•Detecting uneven irrigation/fertilization and necrosis and apoptosis for broccoli cultivation.•Determining the optimal amount of fertilization and the optimal harvest time. Single-plant growth monitoring aids precision agricultural decision-making to reduce the costs related to pesticides, fertilizers, and labor. This study integrated visible/multi-spectral UAV imagery with two deep learning methods, object detection and semantic segmentation, to obtain a visualized map that could assist in precise field monitoring and management for broccoli cultivation. For plant detection, feature extraction was conducted using multiscale dilated convolution, which enabled the effective detection of broccoli in images taken under different photographic conditions and resolutions. Two crops of broccoli (cultivar: Broccoli No. 42) were planted in 2020 at Taichung Agricultural Research and Extension Station, in which the first crop was treated as the training data. The detection of individual broccoli plants was processed using a feature extraction architecture of the AlexNet-Like backend at the SSD frontend, where the input scale of the detector complies with the original SSD architecture. For the model test on the second crop, the recall and precision were 98.58% and 99.73%, respectively, after histogram matching based on the first crop images. Moreover, the proposed approach was applied to a real farming field to verify its robustness across different conditions, and achieved a recall of 61.13% using dilated convolution. This study also generated a visualized growth map on a single-plant basis, which allows operators to detect growth situations, such as uneven irrigation or fertilization and necrosis and apoptosis, to greatly enhance the viability of precision agriculture in the calculation of unit yield and intragroup differences for a regime. The proposed approach can be used to determine the optimal amount of fertilization and observe the size of broccoli heads to determine the optimal harvest time. Expectedly, the method may also be applied to the monitoring and management of other crops to improve the efficiency and reduce the labor demand for precision agriculture.