A systematic assessment of current genome-scale metabolic reconstruction tools

Several genome-scale metabolic reconstruction software platforms have been developed and are being continuously updated. These tools have been widely applied to reconstruct metabolic models for hundreds of microorganisms ranging from important human pathogens to species of industrial relevance. However, these platforms, as yet, have not been systematically evaluated with respect to software quality, best potential uses and intrinsic capacity to generate high-quality, genome-scale metabolic models. It is therefore unclear for potential users which tool best fits the purpose of their research. In this work, we perform a systematic assessment of current genome-scale reconstruction software platforms. To meet our goal, we first define a list of features for assessing software quality related to genome-scale reconstruction. Subsequently, we use the feature list to evaluate the performance of each tool. To assess the similarity of the draft reconstructions to high-quality models, we compare each tool's output networks with that of the high-quality, manually curated, models of Lactobacillus plantarum and Bordetella pertussis, representatives of gram-positive and gram-negative bacteria, respectively. We additionally compare draft reconstructions with a model of Pseudomonas putida to further confirm our findings. We show that none of the tools outperforms the others in all the defined features. Model builders should carefully choose a tool (or combinations of tools) depending on the intended use of the metabolic model. They can use this benchmark study as a guide to select the best tool for their research. Finally, developers can also benefit from this evaluation by getting feedback to improve their software.