Temporal aggregation network using micromotion features for early lameness recognition in dairy cows

•Frame-difference images contain movement features of cows in a short time.•Micromotion features can be obtained using focal convolutional layers.•Multigranularity temporal features are important for early lameness recognition.•This method improves the accuracy of lameness recognition in dairy cows. Lameness severely affects dairy cow welfare and potential milk production while reducing dairy farm economics. Most existing computer vision-based lameness recognition methods extract a single class of lameness features, and it is difficult to accurately recognize early lameness in dairy cows with only slight gait abnormalities. In this paper, a temporal aggregation network using micromotion features is proposed, which can capture micromotion features and spatiotemporal features to recognize early lameness in dairy cows. First, a Gaussian filter is used to remove noise in videos of a dairy cow walking. Second, frame-difference images containing motion features are constructed by the frame-difference method. Third, micromotion features in the frame-difference images are extracted using the designed focal convolutional layer. Fourth, the acquired micromotion feature sequence is input into temporal attention mechanisms to extract multigranularity temporal features. Finally, a fully connected layer is used to classify the dairy cow lameness. To evaluate the performance of the proposed model, 621 videos were randomly selected from 888 videos as training videos, 87 videos were randomly selected as validation videos, and the remaining 180 videos were selected as test videos. The lameness recognition accuracy of the proposed model was 98.89%, which was better than state-of-the-art lameness recognition methods. The experimental results showed that the proposed model can extract the micromotion features and multigranularity temporal features in the frame-difference images, thereby improving the early lameness recognition accuracy in dairy cows.