Does direct observation of procedural skills reflect trainee's progress in otolaryngology?

Background UK surgical trainees are required to undertake work‐based assessments each year in order to progress in their training. Direct Observation of Procedural Skills (DOPS) is one of these assessments. We aim to investigate the validity of DOPS in assessing otolaryngology trainees at all levels. Methods A retrospective search of the portfolios of all otolaryngology trainees in North Thames was carried out to identify otolaryngology‐specific DOPS. A score (Cs) was calculated for each DOPS based on the percentage of satisfactorily‐rated items. The overall performance rating (Ps) was analysed as a separate variable and compared with Cs. The Ps and Cs results were then compared across trainee grades and levels within each grade: Core trainees (CT1‐CT2) and specialty trainees (ST3‐ST8). Results Seven hundred and sixty‐seven otolaryngology DOPS were completed between August 2008 and September 2013. The tool was found to be reliable and internally consistent. Trainees in ST grade had higher Cs and Ps scores than CT grade (P