Reliability of laparoscopic skills assessment on video: 8-year results of the endoscopic surgical skill qualification system in Japan

The Japanese Urological Association and Japanese Society of Endourology established a urologic laparoscopic skills qualification system called the Endoscopic Surgical Skill Qualification (ESSQ) System in Urological Laparoscopy in 2004. The reliability of video assessments by referees was evaluated. Videos of nephrectomies or adrenalectomies performed by the applicants were assessed by two referees selected among a pool of 42 referees. From 2004 to 2011, 1308 urologists applied and 60.2% were qualified after video assessments. The results of skills assessments on 1220 videos that had fixed points by two referees were analyzed statistically. The average number of videos that each referee assessed was 58.1, with a range of 16 to 87. The accordance rate of the results of the video assessment, pass or fail, by the two referees was 68.9%. The scores of the video assessment by each referee averaged 62.7±2.4 (standard deviation) (full score was set at 75 points and ≥60 points was needed to pass). There was a statistically significant difference in the average video assessment score among the referees (P