Deep learning in olive pitting machines by computer vision

•Testing of a neural network based on physical chips to classify olives.•The chipsets are able to process images with 11 × 11 pixels resolution and up to 16 × 16.•The system reaches to 300 olives/min, which is far below the 800–2500 olives/min.•The system offers an alternative to other conventional image classification systems.•The CM1K returns: empty pocket, normal, “boat” position and 255 for untrained cases. Olive pitting machines are characterized by the fact that their optimal functioning is based on an appropriate adjustment: selection of a feed plate adapted to the olive variety and its caliber, geometrical characteristics of the feed chain, etc. The first of these elements sets the optimal way for olives to enter the feed chain and, therefore, it prevents empty pockets or more than one olive to be placed in the same pocket. The second element sets the appropriate position for the olive to be pitted and prevents it to be pitted by a secondary axis. The proposed study analyzes the appropriate placement of olives in the pockets of the feed chain by using the following items:1.A computer vision system with an external trigger, which is capable of taking a picture of every pocket passing in front of the camera.2.A classifying neural network that, appropriately trained, differentiates between four possible pocket cases: empty, normal, incorrectly de-stoned olive in any of its angles (when the olive is de-stoned transversally instead of longitudinally, also known as “boat”) and anomalous case (two olives in the same pocket, small parts of it or foreign elements, such as small branches or stones).A preliminary analysis, carried out with the MATLAB Neural Network Toolbox, has enabled to test the viability of using a neural network to perform this type of classification. The main objective of this paper is to illustrate the use of a physical chip with neural networks, NeuroMem CM1K (General Visions, 2016. CM1K), for sorting purposes. Therefore, it is necessary to identify the minimum resolution required to classify the images of olives in olive pitting machines and their adequate position to be pitted considering an input vector of up to 256 bytes, which is the maximum dimension supported by NeuroMem CM1K. As described before, a camera with an external trigger will be used for image capturing synchronized with the feed chain. Given that the image classification speed must be higher than 15 Hz to be operatively convenient, the industrial feasibility of this s