Development and Validity Evaluation of a National Digital Entrance Examination Test for Higher Education Student Selection

ABSTRACT Higher education institutions need to develop valid, fair, and objective selection methods. Current literature reporting the development and validation of new national large‐scale selection tests is scarce. This two‐phased study aimed to (1) develop and (2) evaluate the validity of the Finnish digital Universities of Applied Sciences Entrance Examination (UAS Exam). In phase 1 (years 2018–2019), the UAS exam sections were constructed using scoping reviews, document analysis, and expert evaluations. Pilot tests were performed for item analysis using Item Response Theory (IRT). In phase 2, validity evaluation was conducted in a cross‐sectional design (October–November 2019) using classical test theory and IRT. The applicants' (n = 13,048) exam scores and the difficulty levels of the exam questions varied in all the exam sections. Psychometrics supported the structure and the use of several exam sections. The usability of the digital exam was evaluated as acceptable. Item analysis revealed development needs in the exam sections.