Performance of Chat GPT on Equilibrium Field of the Otorhinolaryngology Head and Neck Surgery Specialty Certificate Examinations

Chat GPT (Generative Pre-trained Transformer) is a generative artificial intelligence (generative AI) application released by OpenAI in November 2022. It can provide answers in a conversational format, and by specifying the desired response format, one can adjust the output content (such as the word count and expression style). Chat GPT is expected to be used for various applications, such as text summarization, translation, scenario creation, programming, etc. In this study, we presented questions on the vertigo domain from the Otorhinolaryngology Head and Neck Surgery specialist examinations to Chat GPT to evaluate its potential in clinical diagnosis. We presented 51 questions related to vertigo and balance medicine (excluding questions with images) selected from the Otorhinolaryngology specialty certificate examinations held from 2013 (25th) to 2022 (34th) to GPT-3.5 and GPT-4 and evaluated their accuracy rates. The accuracy rates of GPT-3.5 and GPT-4 were 21.6% and 49.0%, respectively. Although the accuracy increased with the version upgrade, Chat GPT is not a specialized AI application for medical use; it is a general-purpose AI. Given the accuracy rates in this study and the possibility of inaccurate answers, it is currently considered uncertain for use for clinical diagnosis in the real world. However, it is expected to be useful for applications such as summarizing patient histories and literature searches to improve the efficiency of medical tasks, and with improvements of the accuracy through version updates, there is potential for application of ChatGPT in the field of medicine in the future.