Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19

To reliably quantify the radiographic severity of COVID-19 pneumonia with the Radiographic Assessment of Lung Edema (RALE) score on clinical chest X-rays among inpatients and examine the prognostic value of baseline RALE scores on COVID-19 clinical outcomes. Hospitalised patients with COVID-19 in dedicated wards and intensive care units from two different hospital systems. 425 patients with COVID-19 in a discovery data set and 415 patients in a validation data set. We measured inter-rater reliability for RALE score annotations by different reviewers and examined for associations of consensus RALE scores with the level of respiratory support, demographics, physiologic variables, applied therapies, plasma host-response biomarkers, SARS-CoV-2 RNA load and clinical outcomes. Inter-rater agreement for RALE scores improved from fair to excellent following reviewer training and feedback (intraclass correlation coefficient of 0.85 vs 0.93, respectively). In the discovery cohort, the required level of respiratory support at the time of CXR acquisition (supplemental oxygen or non-invasive ventilation (n=178); invasive-mechanical ventilation (n=234), extracorporeal membrane oxygenation (n=13)) was significantly associated with RALE scores (median (IQR): 20.0 (14.1-26.7), 26.0 (20.5-34.0) and 44.5 (34.5-48.0), respectively, p