Prudence when assuming normality: An advice for machine learning practitioners

•In binary classification, the input feature vector is given a score to be compared to a threshold for class prediction.•The normal assumption of the score is sometimes severely violated even under the multinormal assumption of the feature vector.•The article proves this mathematically to provide an advice for practitioners to avoid blind assumptions of normality.•On the other hand, the article illustrate some of the expected results of the AUC under multinormal assumption.•Therefore, the message of the article is not to avoid the normal assumption; however, a prudence is needed. [Display omitted] In a binary classification problem the feature vector (predictor) is the input to a scoring function that produces a decision value (score), which is compared to a particular chosen threshold to provide a final class prediction (output). Although the normal assumption of the scoring function is important in many applications, sometimes it is severely violated even under the simple multinormal assumption of the feature vector. This article proves this result mathematically with a counterexample to provide an advice for practitioners to avoid blind assumptions of normality. On the other hand, the article provides a set of experiments that illustrate some of the expected and well-behaved results of the Area Under the ROC curve (AUC) under the multinormal assumption of the feature vector. Therefore, the message of the article is not to avoid the normal assumption of either the input feature vector or the output scoring function; however, a prudence is needed when adopting either of both.