A Further Empirical Investigation of the Relationship Between MRE and Project Size

The mean magnitude of relative error, MMRE, is the de facto standard evaluation criterion to assess the accuracy of software project prediction models. The fundamental metric of MMRE is MRE, a relative residual error. For MMRE to be a meaningful summary statistic, it is a necessary, but not sufficient, condition that MRE and project size are uncorrelated. Except for two previous conference studies done by the same authors, it has never been empirically validated that MRE and project size really are uncorrelated. In this paper, we extend the previous studies using the same data sets as before: Albrecht, Kemerer, Finnish, DMR and Accenture-ERP. Unlike the previous studies, we plot MRE against the predicted effort rather than against the actual effort and, in so doing, we obtain very different results from the previous studies. The results of this study suggest that MRE and project size are uncorrelated, which apparently is contradictory to the previous results where we found a negative correlation. The explanation for these seemingly contradictory results is presented in this study.