A risk of bias instrument for non-randomized studies of exposures: A users' guide to its application in the context of GRADE

The objective of this paper is to explain how to apply, interpret, and present the results of a new instrument to assess the risk of bias (RoB) in non-randomized studies (NRS) dealing with effects of environmental exposures on health outcomes. This instrument is modeled on the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) instrument. The RoB instrument for NRS of exposures assesses RoB along a standardized comparison to a randomized target experiment, instead of the study-design directed RoB approach. We provide specific guidance for the integral steps of developing a research question and target experiment, distinguishing issues of indirectness from RoB, making individual-study judgments, and performing and interpreting sensitivity analyses for RoB judgments across a body of evidence. Also, we present an approach for integrating the RoB assessments within the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework to assess the certainty of the evidence in the systematic review. Finally, we guide the reader through an overall assessment to support the rating of all domains that determine the certainty of a body of evidence using the GRADE approach. •To inform decision-making about environmental and other health exposures, authors of systematic reviews should rigorously and transparently evaluate RoB of the included studies using a standardized approach.•The RoB instrument for NRS of exposures presents a rigorous evaluation tool for RoB assessment for individual studies of environmental exposure for each outcome.•At the review level, overall RoB across the body of evidence for a specific outcome is a crucial part of judging the certainty of that evidence when using the GRADE approach.