Automated segmentation of individual leafy potato stems after canopy consolidation using YOLOv8x with spatial and spectral features for UAV-based dense crop identification

•YOLOv8x with spatial and spectral information is capable for stem segmentation.•0.1 m is a suitable interpolating distance to generate accurate crop height model.•Height mutation is beneficial to segment dense crop.•Millimeter-level ground sampling distance facilitates high throughput phenotyping.•A ratio of 1 for DSM and RGB helps to segment leafy potato stems. High throughput phenotyping of potatoes after canopy consolidation is crucial to crop breeding and management. A prior step is to segment their leafy potato stems, which is challenging after canopy consolidation because potato stems are dense and intertwined. Current methods for dense crop segmentation are manual. This study equipped unmanned aerial vehicles with a high-resolution RGB sensor in ultra-low flight as a high-throughput alternative. An end-to-end method was proposed to segment their leafy potato stems using YOLOv8x and five kinds of band combinations, i.e., RGB, RGB-DSM, RGB-CHM, RGB-DSM × 3, RGB-ExG. The YOLOv8x model with the RGB-DSM combination achieved superior performance with F1 score of 0.86 and Intersection over Union (IoU) of 0.83. Both F1 score and IoU improved by more than 16 %, when adding DSM or CHM to RGB images. Results demonstrated that height mutation at the edge of leafy potato stems played a crucial role in improving the segmentation of leafy potato stems. Millimeter-level ground sampling distance facilitates high throughput phenotyping of potatoes. The accuracy and efficiency of YOLOv8x has great potential for guiding the phenotypic automation of potatoes as well as other arable crops through remote sensing.