Lameness detection of dairy cows based on key frame positioning and posture analysis

•The DeepLabCut model is used to detect 17 skeleton key points in cows.•Key frames are extracted based on the moment the hoof touches the ground.•16 motion features are extracted to detect lameness in dairy cows.•This method improves the robustness of lameness detection. Lameness is one of the common problems on cattle farms and it seriously affects the health and welfare of dairy cows. The existing lameness detection methods fail to accurately locate the key frame of the hoof landing, which makes it difficult to extract the motion features of the dairy cow in a specific posture, such as the front hoof release angle. In this work, the key point detection model is used to locate the key frame, and the multiple motion features in a specific posture are extracted to detect lameness in dairy cows. First, the backbone network of the DeepLabCut model is replaced with a lightweight convolutional neural network EfficientNet-B0 to detect the key points of dairy cows. Second, the backtracking analysis method is used to optimize the key point prediction results, and the video frames of the hoof landing are selected as the key frames according to the coordinate changes of the hoof. Third, according to the key point information contained in the key frames and hoof movement period, the motion features such as front hoof release angle, back arch height, neck tilt angle, support phase and overlap are extracted. Finally, the above features are normalized and input into an SVM model to realize lameness classification. The lameness detection effect of the model was verified by the ten-fold cross-validation method with the videos of 330 different dairy cows. The average accuracy, sensitivity and specificity of lameness detection were 93.64%, 92.74% and 96.89%. In addition, the proposed method was verified in different cattle farm environments, and the experimental results showed that the proposed method has strong robustness.