Privacy-Preserving 3D Skeleton-Based Video Action Recognition via Graph Convolution Network

In recent years, 3D skeleton data-based human action recognition has attracted an increasing number of researchers because of its high robustness under illumination change and scene variation. However, in 3D skeleton data-based human action recognition, there are still some challenges including multi-object recognition and private data leakage. To address this issue, in this paper, we propose a privacy-preserving end-to-end action recognition model based on a Graph Convolutional Network (GCN). Specifically, we innovatively propose a 3D pose estimation method to obtain 3D human skeleton data from color data streams and depth data streams, in which the mean filtering technique is applied to protect users' video data privacy while not affecting the extraction of the skeleton data. Then, we propose a feature representation method and represent the skeleton data to a GCN model, which convolves the centroid of the human skeleton, four limbs, and the trunk of a person. After that, we propose a method to fuse and extract multiple persons' movement features and propose a novel GCN-based interactive action recognition model to recognize multiple people's skeleton data. Moreover, to protect recognition model privacy, we also design a ciphertext-based secure action recognition method, which guarantees the confidentiality of model parameters during action recognition. Finally, we evaluate the performance of our model in real datasets, and the results demonstrate that our model can achieve high recognition accuracy than existing models. Meanwhile, experimental results demonstrate that the mean filtering technique can well protect the privacy of the appearance data and the information of skeleton data can be well preserved.