External Validation of PATHFx Version 3.0 in Patients Treated Surgically and Nonsurgically for Symptomatic Skeletal Metastases

PATHFx is a clinical decision-support tool based on machine learning capable of estimating the likelihood of survival after surgery for patients with skeletal metastases. The applicability of any machine-learning tool depends not only on successful external validation in unique patient populations but also on remaining relevant as more effective systemic treatments are introduced. With advancements in the treatment of metastatic disease, it is our responsibility to patients to ensure clinical support tools remain contemporary and accurate. Therefore, we sought to (1) generate updated PATHFx models using recent data from patients treated at one large, urban tertiary referral center and (2) externally validate the models using two contemporary patient populations treated either surgically or nonsurgically with external-beam radiotherapy alone for symptomatic skeletal metastases for symptomatic lesions. After obtaining institutional review board approval, we collected data on 208 patients undergoing surgical treatment for pathologic fractures at Memorial Sloan Kettering Cancer Center between 2015 and 2018. These data were combined with the original PATHFx training set (n = 189) to create the final training set (n = 397). We then created six Bayesian belief networks designed to estimate the likelihood of 1-month, 3-month, 6-month, 12-month, 18-month, and 24-month survival after treatment. Bayesian belief analysis is a statistical method that allows data-driven learning to arise from conditional probabilities by exploring relationships between variables to estimate the likelihood of an outcome using observed data. For external validation, we extracted the records of patients treated between 2016 and 2018 from the International Bone Metastasis Registry and records of patients treated nonoperatively with external-beam radiation therapy for symptomatic skeletal metastases from 2012 to 2016 using the Military Health System Data Repository (radiotherapy-only group). From each record, we collected the date of treatment, laboratory values at the time of treatment initiation, demographic data, details of diagnosis, and the date of death. All records reported sufficient follow-up to establish survival (yes/no) at 24-months after treatment. For external validation, we applied the data from each record to the new PATHFx models. We assessed calibration (calibration plots), accuracy (Brier score), discriminatory ability (area under the receiver operating characteristic curv