Comparison of convolutional neural networks in fruit detection and counting: A comprehensive evaluation

•Knowing the amount of fruit to be harvested leads to better decisions in agriculture.•Convolutional neural networks (CNN) are the current trend in processing imagery.•There is still missing an insightful analysis of the usability of CNN in fruit counting in groves.•This works presents and systematically tests two CNN architectures in three different groves. Knowing beforehand the amount of fruit to be harvested leads to better logistics and decisions making in the agricultural industry. In the last years, several sensors –mainly artificial vision systems– and sensing techniques have been proposed to address the fruit counting problem with sundry results. Convolutional neural networks (CNN) arise as the current trend in processing imagery information, due to their adaptability and efficiency in object detection. However, there is still missing an insightful analysis of the usability of such technique in fruit counting problems in groves, since the learning process is sensitive to the training input data, the sensor (affected by environmental conditions) and the architecture chosen to process the imagery set. Therefore, in this work we test two of the most common architectures: Faster R-CNN with Inception V2 and Single Shot Multibox Detector (SSD) with MobileNet. These detection architectures were trained and tested on three fruits: Hass avocado and lemon (both from Chile), and apples (from California - USA), under different field conditions. To address the problem of video-based fruit counting, we use multi-object tracking based on Gaussian estimation. Our system achieves fruit counting performances up to 93% (overall for all fruits) using Faster-RCNN with Inception V2, and 90% (overall for all fruits) using SSD with MobileNet. Such results can lead to further improve the decision making process in agricultural practices.