Crowdsourced versus expert evaluations of the vesico-urethral anastomosis in the robotic radical prostatectomy: is one superior at discriminating differences in automated performance metrics?

Crowdsourcing from the general population is an efficient, inexpensive method of surgical performance evaluation. In this study, we compared the discriminatory ability of experts and crowdsourced evaluators (the Crowd) to detect differences in robotic automated performance metrics (APMs). APMs (instrument motion tracking and events data directly from the robot system) of anterior vesico-urethral anastomoses (VUAs) of robotic radical prostatectomies were captured by the dVLogger (Intuitive Surgical). Crowdsourced evaluators and four expert surgeons evaluated video footage using the Global Evaluative Assessment of Robotic Skills (GEARS) (individual domains and total score). Cases were then stratified into performance groups (high versus low quality) for each evaluator based on GEARS. APMs from each group were compared using the Mann–Whitney U test. 25 VUAs performed by 11 surgeons were evaluated. The Crowd displayed moderate correlation with averaged expert scores for all GEARS domains ( r > 0.58, p