ChatGPT-4 Assistance in Optimizing Emergency Department Radiology Referrals and Imaging Selection

The quality of radiology referrals influences patient management and imaging interpretation by radiologists. This study aims to evaluate ChatGPT-4 as a decision support tool for selecting imaging examinations and generating radiology referrals in the emergency department (ED). We retrospectively extracted five consecutive clinical notes from the ED, for each of the following pathologies: pulmonary embolism, obstructing kidney stones, acute appendicitis, diverticulitis, small bowel obstruction, acute cholecystitis, acute hip fracture, and testicular torsion. A total of 40 cases were included. We entered these notes into ChatGPT-4, requesting recommendation on the most appropriate imaging examination and protocol. We also asked the chatbot to generate radiology referrals. Two independent radiologists graded the referral on a 1-5 scale based on clarity, clinical relevance, and differential diagnosis. We compared the chatbot’s imaging recommendations to the American College of Radiology Appropriateness Criteria (ACR AC), and to the examinations performed in the ED. Agreement between readers was assessed using linear weighted Cohen's kappa coefficient. ChatGPT-4's imaging recommendations aligned with ACR AC and ED examinations in all cases. Protocol discrepancies between ChatGPT and the ACR AC were observed in two cases (5%).. ChatGPT-4-generated referrals received mean scores of 4.6 and 4.8 for clarity, 4.5 and 4.4 for clinical relevance, and 4.9 from both reviewers for differential diagnosis. Agreement between readers was moderate for clinical relevance and clarity, and substantial for differential diagnosis grading. ChatGPT-4 has shown potential in aiding imaging study selection for select clinical cases. As a complementary tool, large language models may improve radiology referral quality. Radiologists should stay informed about this technology and be mindful of potential challenges and risks.