A novel multitask transformer deep learning architecture for joint classification and segmentation of horticulture plantations using very High-Resolution satellite imagery

•Novel multitask model for tree density classification and crown segmentation.•DeiT and U-Net integration improves accuracy in satellite image analysis.•Achieves F1 score of 0.91 and mIoU of 0.73 in tree plantation classification.•Ablation study reveals DeiT’s strength in classification, U-Net in segmentation.•Potential application in horticulture and forest ecosystem management. This study introduces MultiTaskDeiTUNet, a novel multitask deep learning architecture designed to tackle the dual challenges of classifying tree plantation densities and segmenting tree crowns in high-resolution (0.7 m) satellite imagery. The core challenge lies in the overlapping spatial patterns of various tree species and densities, complicating the accurate extraction and classification of individual tree crowns. MultiTaskDeiTUNet integrates the Data-efficient Image Transformer (DeiT) with the U-Net model, harnessing DeiT’s strength in contextual detail recognition and spatial dependency capture for density classification, alongside U-Net’s proficiency in capturing low-level features for precise crown segmentation. By addressing the complexities of high-resolution data handling and the simultaneous execution of classification and segmentation tasks, MultiTaskDeiTUNet achieves an average F1 score of 0.91 (± 0.03) and a precise tree crown segmentation with mIoU of 0.73 (± 0.01). The DeiT backbone adeptly learns shared features such as canopy shapes and spatial arrangements, which are crucial for both tasks and enhance overall model performance. Ablation studies underscore the specialized roles of each component: freezing DeiT’s weights results in reduced classification accuracy with an average F1 score of 0.48 (± 0.08), while freezing U-Net’s weights yields a reduced mIoU of 0.29 (± 0.12) This differentiation highlights DeiT’s excellence in classification tasks and U-Net’s superiority in segmentation. Substituting the DeiT model with a standard ViT model further highlights the effectiveness of DeiT, as the ViT model demonstrated lower accuracy, with an average F1 score of 0.87 (±0.05) compared to DeiT’s F1 score of 0.91 (±0.03). Statistical analysis revealed right-skewed distributions in tree crown areas across density categories. The efficacy of MultiTaskDeiTUNet in tree plantation analysis indicates its potential applicability to a wide range of horticultural plants. Customizing the architecture to species-specific characteristics and varying image resolutions could provide va