YOLO-FD: An accurate fish disease detection method based on multi-task learning

•Integrating a novel semantic segmentation branch into the YOLOv8 network.•Proposing a multi-task learning network YOLO-FD for detection and segmentation.•Weight uncertainty and PCGrad are employed to YOLO-FD.•Achieving accurate fish disease detection and assessment of fish disease severity. Fish diseases often exhibit high risks of contagion, resulting in substantial economic losses. Accurate assessment of fish disease severity during diagnosis using deep learning poses a considerable challenge. Currently, deep learning models mainly focus on single tasks in fish disease detection, such as classification, object detection and segmentation. However, the accurate assessment of fish disease severity requires the integration of multiple dimensions of information, which is beyond the capabilities of traditional single-task methods. Therefore, this paper proposes YOLO-FD, a multi-task learning network specifically designed for simultaneous detection and segmentation. YOLO-FD extends the YOLOv8 backbone by integrating a novel semantic segmentation branch dedicated to precisely segmenting infected areas in diseased fish, while retaining the original object detection branch for identifying infected fish. Weight uncertainty and PCGrad are employed to balance the weights of different losses and to optimize conflicting gradients during the training process. With only a negligible increase in network parameters, YOLO-FD, tested on our constructed Nocardiosis fish dataset, achieves a detection accuracy of 94.2% mAP50 and gets mIOU of 79.4%, showcasing a 0.5% improvement over the baseline YOLOv8 and surpassing the state-of-the-art semantic segmentation network Deeplabv3plus by 4%. Notably, compared to the adapted multi-task network YOLOP, YOLO-FD demonstrates substantial improvements, displaying a 13.7% increase in mAP50-95 and a 15.1% boost in mIOU. On the VOC2012 segmentation dataset, the proposed method exhibits a 3.2% increase in mAP50 and a 2.2% rise in mAP50-95 compared to the baseline. Furthermore, results of the ablation experiment validate the effectiveness and generalization of the proposed multi-task learning approach. Source code is available at https://github.com/feifei-Lee/YOLO-FD.