Vision-based estimation of MDS-UPDRS scores for quantifying Parkinson's disease tremor severity

•We propose a video-based method for estimating PD tremor severity scores using tremor videos and classify them into clinical scores.•EVM was innovatively used for the pre-processing of tremor video to magnify subtle tremors in the video.•Our proposed GSM has a significant superiority in the identification of tremors compared to other methods.•Our method (EVM+GTSN) outperforms other state-of-the-art models in making score prediction on a fairly large dataset. Parkinson's disease (PD) is a common neurodegenerative movement disorder among older individuals. As one of the typical symptoms of PD, tremor is a critical reference in the PD assessment. A widely accepted clinical approach to assessing tremors in PD is based on part III of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS). However, expert assessment of tremor is a time-consuming and laborious process that poses considerable challenges to the medical evaluation of PD. In this paper, we proposed a novel model, Global Temporal-difference Shift Network (GTSN), to estimate the MDS-UPDRS score of PD tremors based on video. The PD tremor videos were scored according to the majority vote of multiple raters. We used Eulerian Video Magnification (EVM) pre-processing to enhance the representations of subtle PD tremors in the videos. To make the model better focus on the tremors in the video, we proposed a special temporal difference module, which stacks the current optical flow to the result of inter-frame difference. The prediction scores were obtained from the Residual Networks (ResNet) embedded with a novel module, the Global Shift Module (GSM), which allowed the features of the current segment to include the global segment features. We carried out independent experiments using PD tremor videos of different body parts based on the scoring content of the MDS-UPDRS. On a fairly large dataset, our method achieved an accuracy of 90.6% for hands with rest tremors, 85.9% for tremors in the leg, and 89.0% for the jaw. An accuracy of 84.9% was obtained for postural tremors. Our study demonstrated the effectiveness of computer-assisted assessment for PD tremors based on video analysis. The latest version of the code is available at https://github.com/199507284711/PD-GTSN. [Display omitted]