Application of an image-based head detection method for yield trial plots in wheat and barley breeding programs

In wheat and barley breeding, field phenotyping requires a lot of time and effort, so speeding up and automating these processes play an important role. Image sensing technology, which has developed dramatically with the advent of deep learning methods, makes it possible to acquire various types of information from images quickly with high precision. In this study, we aimed to improve the efficiency of breeding using such image sensing technology. At the first step, we attempted to develop a method for head detection and counting using images of yield trail plots in wheat and barley breeding programs. For developing the method, we used YOLOv4 and created a model using 2,023 training images and 674 validating images for three post-flowering stages. The developed model showed good accuracy with an mAP (mean Average Precision) of 85.13% for untrained data, considered to be robust for images of different wheat and barley types and ripening stages. Using the detection model combined with tracking technology, we attempted to estimate the number of heads from consecutive video frames. By using the output of the model, we tested two types of calculation methods for counting heads: the average number of heads per frame and the total number of unique heads in the video, while changing the detection threshold. As a result, the number of heads based on the total number of unique heads when the threshold was set at 0.35 showed a high correlation with the actual values, with coefficients of determination of 0.726 for barley and 0.510 for wheat. When the estimated number of heads from images was compared with the values obtained by conventional visual measurement, the average correlation coefficient over two years was 0.499 for barley and 0.337 for wheat. Since the method developed in this study is simpler than the conventional method and has excellent reproducibility between replications, it can save labor, and speed up and provide high accuracy in head count surveys of wheat and barley breeding programs.