Development of a novel behaviourally anchored instrument for the assessment of surgical trainees

Background The Royal Australasian College of Surgeons (RACS) created its competency framework in 2003 which initially consisted of nine competencies each regarded as equally important for a practising surgeon. The JDocs Framework is aligned to these competencies and provides guidance for junior doctors working towards the Surgical Education and Training program. Methods A novel assessment instrument was designed around the JDocs framework using 48 behaviourally anchored questions. The study was completed in 2020 across five public hospitals in the ACT and NSW. Participants were invited to complete the self‐assessment form online. Results Thirty‐six of 59 (61%) trainees participated in the study, with 67 of 68 (98.5%) supervisors having completed the assessment form. Trainee self‐rating scores were lower than that of supervisor ratings across all competencies except communication. The self‐rating scores were negatively correlated with the seniority of a trainee's level in all nine competencies. The years of post‐graduate experience was positively correlated with seven of the nine competencies. For gender and International Medical Graduate status, correlation was only identified for health advocacy and medical expertise. There was no correlation identified with a trainee's age. Conclusion This pilot study has provided an opportunity to explore a new assessment instrument for surgical trainees that is aligned to the RACS competency framework using behaviourally anchored questions. Looking ahead, a better understanding of this instrument will potentially be helpful in early identification of underperforming trainees in order to facilitate early intervention, or its use as a selection tool for formal training programs. A novel assessment instrument was designed around the JDocs framework using 48 behaviourally anchored questions. Trainees and supervisors participated in the study and we identified the factors that correlate with scoring higher or lower in each of the nine surgical competencies. This pilot study has provided an opportunity to explore a new assessment instrument for surgical trainees and will potentially be helpful in early identification of underperforming trainee, or its use as a selection tool for formal training programs.