Development of a large urban longitudinal HIV clinical cohort using a web-based platform to merge electronically and manually abstracted data from disparate medical record systems: technical challenges and innovative solutions

Electronic medical records (EMRs) are being increasingly utilized to conduct clinical and epidemiologic research in numerous fields. To monitor and improve care of HIV-infected patients in Washington, DC, one of the most severely affected urban areas in the United States, we developed a city-wide database across 13 clinical sites using electronic data abstraction and manual data entry from EMRs. To develop this unique longitudinal cohort, a web-based electronic data capture system (Discovere®) was used. An Agile software development methodology was implemented across multiple EMR platforms. Clinical informatics staff worked with information technology specialists from each site to abstract data electronically from each respective site's EMR through an extract, transform, and load process. Since enrollment began in 2011, more than 7000 patients have been enrolled, with longitudinal clinical data available on all patients. Data sets are produced for scientific analyses on a quarterly basis, and benchmarking reports are generated semi-annually enabling each site to compare their participants' clinical status, treatments, and outcomes to the aggregated summaries from all other sites. Numerous technical challenges were identified and innovative solutions developed to ensure the successful implementation of the DC Cohort. Central to the success of this project was the broad collaboration established between government, academia, clinics, community, information technology staff, and the patients themselves. Our experiences may have practical implications for researchers who seek to merge data from diverse clinical databases, and are applicable to the study of health-related issues beyond HIV.