Science teachers' TPACK-Practical: Standard-setting using an evidence-based approach

Technological pedagogical content knowledge-practical (TPACK-P) refers to the knowledge construct that teachers in the digital era develop for and from their teaching practices with technology. This study explored a standard-setting method using item response theory to cross-validate ranks of proficiency levels and examine in-service and pre-service science teachers' knowledge about and application of TPACK-P in Taiwan. A sample of 99 participants (52 pre-service and 47 in-service science teachers) completed a 17-item TPACK-P questionnaire that described their typical responses, opinions, or actions in different instructional scenarios. Initial analysis of these responses revealed a correlation (r = 0.87) between the ranks of proficiency levels and those previously identified that validated the hierarchical structure of the four proficiency levels (1-lack of use, 2-simple adoption, 3-infusive application, and 4-reflective application). The second analysis located the thresholds of the four proficiency levels in the two dimensions of knowledge about and application of TPACK-P. It was found that there were no significant differences between in-service and pre-service teachers' TPACK-P and that most of the participants displayed knowledge about TPACK-P at Levels 2 and 3, but their application was at Level 1. The validated four proficiency levels coupled with typical performances can be viewed as a roadmap of science teachers' TPACK-P development. The gap between the knowledge about and application of TPACK-P suggests that further practical experiences in supportive environments are needed in science teacher education programs. Only when teachers gain and learn from practical usage of technology to support science education can their TPACK-P be further developed and strengthened. •Benchmarks of TPACK-P by proficiency level were identified from interview results.•Four proficiency levels and associated features were validated.•IRT was used to locate the thresholds of TPACK-P proficiency levels.•Science teachers displayed higher levels of knowledge than application of technology.