Nailed it! Doctor of pharmacy learners' self-awareness of performance on authentic assessments

The primary objective of this pilot study was to determine the reliability and validity of an objective structured clinical examination (OSCE) self-awareness survey. A secondary objective was to assess learner self-awareness in OSCE performance. A retrospective review of OSCE data from a single cohort in a three-year accelerated doctor of pharmacy program occurred. The cohort completed four OSCEs with a total of 23 discrete cases. At the end of each case, before receiving feedback, learners completed an evaluation of their skills, communication, overall performance, and identified if they believed the patient/provider would return (“nailed or failed”). These self-evaluations were then compared with respective learner performance scores for each case. Content validity was assessed for the self-awareness survey via the Lawshe method. Reliability of the OSCE cases and self-awareness survey were conducted utilizing Cronbach's alpha and the Spearman-Brown formula, respectively. The OSCE self-awareness survey was found to be face valid and reliable for the overall survey. The subsections of return to pharmacist and overall learner performance were also found to be valid and reliable. A statistical relationship was found between learner self-assessment of pass/fail and their actual grade for overall performance and return to pharmacist. Results from this study may guide remediation plans related to OSCE performance and development of other self-awareness activities in practice settings. Further studies are needed to evaluate the broader application and use of self-awareness tools.