A pareto-based ensemble of feature selection algorithms

•We have designed a method for ensemble feature selection.•We model the feature selection process to a Pareto-based optimization problem.•The crowding distance between solutions is the secondary measure.•The method is an ensemble of relevancy and redundancy methods.•The proposed PEFS method outperforms competitive algorithms. In this paper, ensemble feature selection is modeled as a bi-objective optimization problem regarding features’ relevancy and redundancy degree. The proposed method, which is called PEFS, first uses the modeled bi-objective optimization problem to find the non-dominated features based on the decision matrix constructed by different feature selection algorithms. In the second step, the found non-dominated features are sorted using the crowding distance in the bi-objective space. These sorted features remove from the feature space, and the process of finding the non-dominated features will continue until all the features are sorted. To illustrate the optimality and efficiency of the proposed method, we have compared our approach with some ensemble feature selection methods and basic algorithms used in the ensemble process. The results show that our method in terms of accuracy and F-score is superior to other similar methods and performs in a short running-time.