Competency assessors’ cognitive map of practice when assessing practice based encounters

Introduction: There is growing evidence that inconsistencies exist in how competencies are conceptualised and assessed. Aim: This study aimed to determine the reliability of pharmacist assessors when observing practice-based encounters and to compare and contrast assessors’ cognitive map of practice with the guiding competency framework. Methods: This was a qualitative study with verbal protocol analysis. A total of 25 assessors were recruited to score and verbalise their assessments for three videos depicting practice-based encounters. Verbalisations were coded according to the professional competency framework. Results: Protocols from 24 participants were included. Interrater reliability of scoring was excellent. Greater than 75% of assessment verbalisations were focused on 3 of the 27 competencies: communicate effectively, consults with the patient, and provide patient counselling. Conclusion: Findings support the notion that assessment completed within practice could be largely informed by a single component of the interaction or more specifically, what ‘catches the eye’ of the assessor.