Battle of the bots: a comparative analysis of ChatGPT and bing AI for kidney stone-related questions
Objectives To evaluate and compare the performance of ChatGPT™ (Open AI ® ) and Bing AI™ (Microsoft ® ) for responding to kidney stone treatment-related questions in accordance with the American Urological Association (AUA) guidelines and assess factors such as appropriateness, emphasis on consultin...
Gespeichert in:
| Veröffentlicht in: | World journal of urology 2024-10, Vol.42 (1), p.600, Article 600 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Zusammenfassung: | Objectives
To evaluate and compare the performance of ChatGPT™ (Open AI
®
) and Bing AI™ (Microsoft
®
) for responding to kidney stone treatment-related questions in accordance with the American Urological Association (AUA) guidelines and assess factors such as appropriateness, emphasis on consulting healthcare providers, references, and adherence to guidelines by each chatbot.
Methods
We developed 20 kidney stone evaluation and treatment-related questions based on the AUA Surgical Management of Stones guideline. Questions were asked to ChatGPT and Bing AI chatbots. We compared their responses utilizing the brief DISCERN tool as well as response appropriateness.
Results
ChatGPT significantly outperformed Bing AI for questions 1–3, which evaluate clarity, achievement, and relevance of responses (12.77 ± 1.71 vs. 10.17 ± 3.27;
p
|
|---|---|
| ISSN: | 1433-8726 0724-4983 1433-8726 |
| DOI: | 10.1007/s00345-024-05326-1 |