Potato Detection and Segmentation Based on Mask R-CNN

Purpose Potatoes are similar in color and size to soil and its clods. They are mostly irregular in the shape as well. Therefore, it is not easy to distinguish potatoes from the soil surface background only with machine vision. This study applied Mask R-CNN, one of the object recognition technologies using deep learning to detect potatoes. The size of object in pixel was obtained on individual potato, and they will be used to predict the yield of potatoes. Methods In order to collect the images needed for deep learning, potato images at the time of harvesting were obtained from potato farms. Annotation was entered for each irregular potato shape, where approximately 4500 potatoes were used. Resnet-101 was selected as the backbone of Mask R-CNN with a feature pyramid network. Transfer training was applied to shorten training time and limit the number of images needed to train the model. The classification performance evaluation was conducted to verify the trained model. The size of potato in pixel was obtained from the output image through the potato detection model by the segmentation algorithm using MATLAB. Results The total number of training for the potato detection model was 12,000, and the training loss of Mask R-CNN was less than 0.1%. The potato detection results from 69 randomly selected test images showed that the average detection precision was 90.8%, recall 93.0%, and F1 score 91.9%. Conclusions Potato detection model with Mask R-CNN can detect irregularly shaped potatoes on similar color soil surface. The size of the detected potato region can be extracted as well.