Using the payback framework to evaluate the outcomes of pilot projects supported by the Georgia Clinical and Translational Science Alliance

The Clinical and Translational Science Awards (CTSA) program of the National Center for Advancing Translational Sciences (NCATS) seeks to improve population health by accelerating the translation of scientific discoveries in the laboratory and clinic into practices for the community. CTSAs achieve this goal, in part, through their pilot project programs that fund promising early career investigators and innovative early-stage research projects across the translational research spectrum. However, there have been few reports on individual pilot projects and their impacts on the investigators who receive them and no studies on the long-term impact and outcomes of pilot projects. The Georgia CTSA funded 183 pilot projects from 2007 to 2015. We used a structured evaluation framework, the payback framework, to document the outcomes of 16 purposefully-selected pilot projects supported by the Georgia CTSA. We used a case study approach including bibliometric analyses of publications associated with the selected projects, document review, and investigator interviews. These pilot projects had positive impact based on outcomes in five "payback categories": (1) knowledge; (2) research targeting, capacity building, and absorption; (3) policy and product development; (4) health benefits; and (5) broader economic benefits. Results could inform our understanding of the diversity and breadth of outcomes resulting from Georgia CTSA-supported research and provide a framework for evaluating long-term pilot project outcomes across CTSAs.