Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source

Background Large language models (LLM) have unknown implications for medical research. This study assessed whether LLM‐generated s are distinguishable from human‐written s and to compare their perceived quality. Methods The LLM ChatGPT was used to generate 20 arthroplasty s (AI‐generated) based on full‐text manuscripts, which were compared to originally published s (human‐written). Six blinded orthopaedic surgeons rated s on overall quality, communication, and confidence in the authorship source. Authorship‐confidence scores were compared to a test value representing complete inability to discern authorship. Results Modestly increased confidence in human authorship was observed for human‐written s compared with AI‐generated s (p = 0.028), though AI‐generated authorship‐confidence scores were statistically consistent with inability to discern authorship (p = 0.999). Overall quality was higher for human‐written s (p = 0.019). Conclusions AI‐generated s' absolute authorship‐confidence ratings demonstrated difficulty in discerning authorship but did not achieve the perceived quality of human‐written s. Caution is warranted in implementing LLMs into scientific writing.