Development of a scoring model for the Sharp/van der Heijde score using convolutional neural networks and its clinical application

Abstract Objectives To construct a predictive model for the Sharp/van der Heijde score (SHS) and assess its applicability in clinical research settings. Material and methods A prediction model for SHS was constructed in three steps using convolutional neural networks (CNN) and an in-house RA image database: orientation, detection and damage prediction. A predictive model for radiographic progression (ΔSHS >3/year) was developed using a graph convolutional network (GCN). A multiple regression model was used to assess the association between predicted SHS using the CNN model and clinical features. Results In the orientation and detection phases, 100% accuracy was achieved in the image orientation correction, and all predicted joint coordinates were within 10 pixels of the correct coordinates. In the damage prediction phase, the κ values between the model and expert 1 were 0.879 and 0.865 for erosion and joint space narrowing, respectively. Using a dataset scored by experts 1 and 2, a minimal overfitting was determined to the scoring by expert 1. High-titre RF was an independent risk factor of ΔSHS per year, as predicted by the CNN model in biologics users. The AUCs of the GCN model for predicting ΔSHS >3/year in patients with and without biologics at baseline were 0.753 and 0.734, respectively, superior to those of the other models. The RF titre was the most important feature in predicting ΔSHS >3/year in biologics users in the GCN model. Conclusion A high-performance scoring model for SHS that is applicable to clinical research was constructed. Graphical Abstract