Validating Chemistry Faculty Members’ Self-Reported Familiarity with Assessment Terminology

With the increasing emphasis placed upon chemistry instructors and departments to assess and evaluate their courses and curricula, understanding the structure of chemistry faculty members’ knowledge and use of assessment terms and concepts can shed light on potential areas for targeted professional development. Survey research that might accomplish this objective often relies on self-reported responses from the target audience, and such information is sometimes difficult to assess in terms of validity. As an example of an internal mechanism to help establish validity, it is possible to include an “internal standard” item early in the survey. For the sake of understanding faculty members’ familiarity with assessment terminology, an item that asked participants to identify analogous pairs of terms comparing assessment measures (assessment validity and assessment reliability) to laboratory measures (accuracy and precision) served this purpose. Using ordered logistic regression, participants who answered the analogy question completely correctly were more likely to report higher levels of familiarity with the assessment terms. Because the self-reported data appears to be valid, these data can be further used in subsequent analyses in order to determine the general familiarity trends among chemistry faculty regarding assessment terminology.