Automated detection and analysis of piglet suckling behaviour using high-accuracy amodal instance segmentation

•GCN-assisted and boundary-guided high-accuracy amodal instance segmentation;•Robust instance segmentation against occlusion and adhesion between piglets;•High-quality motion features describing continuity and reciprocation of piglet suckling behavior;•Accurate analysis of the number of suckling piglets in a suckling event. Piglet suckling behaviour is a critical indicator of piglet liveability and health status; however, automated detection of this important behaviour is rarely reported in literatures. In this study, we proposed a two-step computer vision-based detection method for piglet suckling behaviour. In the first step, an anchor-free deep learning network was employed in instance segmentation of individual sows and piglets. Firstly the localization head detected piglets to obtain the features of the region of interest (ROI). The features extracted from ROI were passed to a novel attention graph convolution-based structure to distil element-wise features. The distilled features were further encoded by a graph convolutional network and were fed into the boundary head and the mask head for piglet contour and mask prediction, respectively. In the second step, the piglets, in adhesion with the sow, were tracked by intersection over union (IOU) which was calculated between adjacent frames. Piglet motion features were derived from the maximum, minimum, variance, and average values in IOU sequence within the 21-frame (3-s) independent processing units. The extracted motion features were input into an SVM, classifying a piglet into suckling or nonsuckling. The dataset, for training and verifying the proposed network respectively, was composed of 100 1-min and 7-fps short video clips as well as one 8-h long video episode, from seven pens of Large White sows and piglets. Our method achieved favourable detection performance with F1 score of 93.6%, Recalls of 92.1%, and Precisions of 95.2% in short video clips, which showed that detecting suckling behaviours for piglets using amodal instance segmentation was feasible. The time budgets of at least one piglet, more than half of the piglets, or all piglets exhibiting suckling behaviour were 74.0 min, 65.6 min, and 1.1 min in an 8-h long video.