HoopTransformer: Advancing NBA Offensive Play Recognition with Self-Supervised Learning from Player Trajectories

Background and Objective Understanding and recognizing basketball offensive set plays, which involve intricate interactions between players, have always been regarded as challenging tasks for untrained humans, not to mention machines. In this study, our objective is to propose an artificial intelligence model that can automatically recognize offensive plays using a novel self-supervised learning approach. Methods The dataset was collected by SportVU from 632 games during the 2015–2016 season of the National Basketball Association (NBA), with a total of 90,524 possessions. A multi-agent motion prediction pretraining model was built on the basis of axial-attention transformer and trained with different masking strategies: motion prediction (MP), motion reconstruction (MR), and MP + MR joint strategy. A downstream play-level classification task and similarity search were used to evaluate the models’ performance. Results The results showed that the MP + MR joint masking strategy maximized the ability of the model compared with individual masking strategies. For the classification task, the joint strategy achieved a top-1 accuracy of 81.5% and top-3 accuracy of 97.5%. In the similarity search evaluation, the joint strategy attained a top-5 accuracy of 76% and top-10 accuracy of 59%. Additionally, with the same MP + MR joint masking strategy, our HoopTransformer model outperformed the two baseline models in the classification task and similarity search. Conclusion This study presents a self-supervised learning model and demonstrates the effectiveness and potential of the model in accurately comprehending and capturing player movements and complex interactions during offensive plays.