Toward the translation of systems thinking methods in patient safety practice: Assessing the validity of Net-HARMS and AcciMap

•It is not clear whether contemporary safety methods work as intended when used by healthcare practitioners.•We assessed the criterion-referenced validity of Net-HARMS and AcciMap.•67 healthcare practitioners applied both methods on two occasions.•Net-HARMS achieved poor levels of validity, whereas AcciMap achieved high levels.•Further work is required to support the use of state-of-the-art safety science methods in practice. In healthcare practice, state-of-the-art safety science methods are overlooked in favour of established methods. Bridging this ‘research-practice’ gap is critical to support safe, high-quality healthcare; however, it is not clear whether contemporary methods work as intended when used by practitioners. This study aimed to assess the criterion-referenced validity of systems thinking-based risk assessment (Net-HARMS) and accident analysis methods (AcciMap). Sixty-seven healthcare practitioners from Queensland, Australia, received training in each method and used Net-HARMS to identify risks associated with patient medication administration tasks and AcciMap to analyse an adverse medication administration event. Validity was assessed by comparing participants’ analyses to ‘gold standard’ expert analyses. Net-HARMS achieved poor levels of validity for both task and emergent risk identification. AcciMap achieved high levels of validity for the placement of contributory factors at the correct healthcare system level but only moderate levels when identifying relationships between contributory factors. The findings highlight the need for further work to support the use of state-of-the-art safety science methods in practice. This includes the provision of education in systems thinking, methods training programs, the development of further support materials, and modification of the methods to improve their usability.