Segmentation of abnormal leaves of hydroponic lettuce based on DeepLabV3+ for robotic sorting

•Segmenting abnormal leaves for hydroponic lettuce sorting using DeepLabV3+ model.•Xception-65, Xception-71, ResNet-50, and ResNet-101 were trained and compared.•Median frequency weight was used to mitigate data imbalance.•ResNet-101 achieved the highest mIoU and acceptable speed for automatic sorting.•Results provided guidance for automatic sorting device of hydroponic lettuce. Hydroponic lettuce has been widely cultivated in plant factory and desiring for mechanical harvesting and packing. Sorting of hydroponic lettuce must be carried out before packing. Information perception and image processing of hydroponic lettuce is a crucial technology to develop a robotic sorting system. In this study, DeepLabV3+ models of deep learning technologies were employed with four backbones of ResNet-50, ResNet-101, Xception-65, and Xception-71 to design a vision system of segmenting abnormal leaves (yellow, withered, and decay leaves) of hydroponic lettuce. Two weights assignation methods, i.e., median frequency weights (MFW) and uniform weights (UW), were incorporated into DeepLabV3+ and compared for performance. Results showed that models trained by UW were better than that of MFW assignation method. ResNet-101 had the best segmentation performance in UW assignation method with pixel accuracy of 99.24% and mIoU of 0.8326. In terms of speed, ResNet-50 had the fast segmentation speeds with 154.0 ms per image. This study provided object detection methodology for automatic sorting device of hydroponic lettuce.