A risk and credibility framework for in silico clinical trials of medical devices

•In silico clinical trials (ISCTs) are an emerging application for simulation.•Appropriate model credibility via clinical validation activities is needed.•A framework is developed for evaluating model risk associated with an ISCT.•Credibility factors specific to clinical validation activities are proposed.•Supplementary material applies the framework to a shoulder arthroplasty case study. The use of in silico clinical trials (ISCTs) to generate clinically-relevant data on new medical devices is an emerging area of regulatory research. Interest in ISCTs stems from recognized challenges in acquiring sufficient clinical data and the continued maturation of in silico technologies. There is currently no guidance in place for evaluating the credibility of ISCT applications. The objective of this work was to adapt an existing risk-based credibility framework specifically for ISCT applications, and demonstrate its utility on a contemporary case study. Expanding on guidance currently in place for assessing the risk of traditional modeling applications of medical devices and demonstrating model credibility through benchtop validation activities, a framework is proposed to (1) evaluate the model risk for ISCT applications based on the independent factors of scope, coverage, and severity, and (2) assess the credibility of clinical validation activities based on consideration of the clinical comparator, the validation model, the agreement between the two, and the applicability of the clinical validation activities to the ISCT application. The resulting framework spans across the range of ISCT applications that may be envisioned, as well as the variety of clinical datasets that can be used to demonstrate model credibility. Credibility factors reflect the expected clinical variability in the validation comparator and validation model, the statistical power of the comparator, the rigor of agreement between the comparator and model in terms of both inputs and outputs, and the overall similarity of the device in the validation activities to the device within the intended ISCT. When applied to a high-risk case study, the framework reveals that planned clinical validation activities require additional rigor in order to achieve the credibility targets, enabling an assessment of the validation effort relative to the potential benefit prior to investing in the validation studies. An objective and risk-based framework for establishing credibility requirements for ISCT application