Evidence-based evolution of the high-stakes postgraduate intensive care examination in Australia and New Zealand

The Fellowship Examination for Intensive Care Medicine in Australian and New Zealand, first held in 1979, has undergone four distinct periods of development and change: * 1979-1996. Initiation and establishment of the exam as a relevant and comprehensive assessment process for a new specialty. * 1997-2001. (*) Revision to increase breadth of coverage and reliability for a growing number of candidates, and to ensure that each candidate received the same exam. (*) Expansion to incorporate assessment of CanMEDS skills, including communication, procedures and professional qualities. (*) Lengthening to increase the number of exposures, to ensure reliability. (*) Quarantining of candidates to allow the provision of a similar exam for each candidate. * 2002-2006. Increasing emphasis on examiner training and standard setting, increasing feedback to candidates to improve the educational experience and guide exam preparation, and blueprinting of questions to maintain validity. * 2007 onwards. Logistic revision to ensure feasibility for a rapidly growing number of candidates, and refinement to apply modern standard setting and quality control. The exam has been regarded as a "tough but fair" assessment in its 30 years of existence, and the committee overseeing its development has aimed to continually review the process to maintain those qualities, as well as reliability, validity and feasibility. The increasing number of candidates has allowed usable statistics to be accumulated but has tested the feasibility of running such a labour-intensive exam. To date, there have been 800 presentations to the exam, with 498 successful candidates. (author abstract)