Neural network for ordinal classification of imbalanced data by minimizing a Bayesian cost

•An estimate of the Bayes cost is proposed as the loss to train neural networks for ordinal classification of imbalanced data.•The network parameters, as well as the decision thresholds, are updated during training to minimize the Bayes cost.•The neural network architecture has a single neuron in the output layer (one-dimensional input space).•Both shallow networks and deep networks can be used.•Experiments with real data show the accuracy and flexibility of the proposed method, specially in imbalanced problems. Ordinal classification of imbalanced data is a challenging problem that appears in many real world applications. The challenge is to simultaneously consider the order of the classes and the class imbalance, which can notably improve the performance metrics. The Bayesian formulation allows to deal with these two characteristics jointly: It takes into account the prior probability of each class and the decision costs, which can be used to include the imbalance and the ordinal information, respectively. We propose to use the Bayesian formulation to train neural networks, which have shown excellent results in many classification tasks. A loss function is proposed to train networks with a single neuron in the output layer and a threshold based decision rule. The loss is an estimate of the Bayesian classification cost, based on the Parzen windows estimator, which is fitted for a thresholded decision. Experiments with several real datasets show that the proposed method provides competitive results in different scenarios, due to its high flexibility to specify the relative importance of the errors in the classification of patterns of different classes, considering the order and independently of the probability of each class.