Multimodal Neural Network for Recurrence Prediction of Papillary Thyroid Carcinoma

Papillary thyroid carcinoma (PTC) is the most common endocrine carcinoma and has frequent recurrence instances. Although PTC recurrence has been predicted using predictors established using various features and techniques, its early detection is still challenging. To address this issue, it is aimed to develop a deep‐learning model that utilizes not only the initial medical records but also the thyroid function tests (TFTs) performed periodically post‐surgery. Herein, a novel multimodal prediction model, called the hybrid architecture for multimodal analysis (HAMA), that can analyze numeric and time‐series data simultaneously, is proposed. For quantitative evaluation, fourfold cross validation is conducted on data of 1613 PTC patients including 63 locoregional recurrence patients, and the HAMA is achieved the following performance: sensitivity (0.9688); specificity (0.9781); F1‐score (0.7943); and area under the receiver‐operating characteristic curve, AUROC (0.9863). Furthermore, a real‐time prediction simulation is conducted at 6‐month intervals by reconstructing the data of each patient into real‐time data. It is demonstrated in the real‐time simulation results that the HAMA predicts PTC recurrence at least 1.5 years in advance by recalculating the recurrence probability using the additional follow‐up data. To the best of the knowledge, the HAMA is the first deep‐learning model to reflect continuous change in the physical condition of a patient post‐surgery. The hybrid architecture for multimodal analysis (HAMA) is a novel deep‐learning model designed to predict papillary thyroid carcinoma (PTC) recurrence. Herein, a high diagnostic accuracy of 98% is demonstrated by the HAMA by simultaneously analyzing data at the time of PTC surgery and thyroid function tests (TFTs) data. Surprisingly, the HAMA can predict PTC recurrence from 1.5 years in advance.