National Emergency Department Overcrowding Study tool is not useful in an Australian emergency department

Objective: To determine the accuracy and usefulness of the National Emergency Department Overcrowding Study (NEDOCS) tool in an urban hospital ED in Australia by direct comparison with subjective assessment by senior ED staff. Method: A sample of simultaneous subjective and objective data pairs were collected six times a day for a period of 3 weeks. All senior medical staff in the ED answered a brief questionnaire along with the senior charge nurse for the ED. Simultaneously, the senior charge nurse also documented the total number of patients in the ED, the number of patients awaiting admission, the number of patients on ventilators, the longest time waited by an ED patient for ward bed, and the waiting time for the last patient from the Waiting Room placed on a trolley. The objective indicators were entered into a Web‐based NEDOCS tool and transformed scores were compared with the averaged and transformed subjective scores for each sample time. Bland–Altmann and Kappa statistics were used to test the agreement between the objective and subjective measuring methods. Results: The mean difference between the subjective and objective methods was small (3.5 [95% confidence interval −0.875–7.878] ); however, the 95% limits of agreement was wide (−46.52–53.43). The Kappa statistic used to assess the extent of reproducibility between categorical variables was 0.31 (95% confidence interval 0.17–0.45). Conclusion: The present study suggests that NEDOCS method of processing the objective overcrowding data does not accurately reflect the subjective assessment of the senior staff working at that time in the ED. This might be because the assumptions of the original NEDOCS study are flawed.