A graph based preordonnances theoretic supervised feature selection in high dimensional data

Generally, for high-dimensional datasets, only some features are relevant, while others are irrelevant or redundant. In the machine learning field, the use of a strategy for eliminating insignificant features from a dataset is very important for the classification task. Feature selection is the process of identifying the most informative features that help in predicting sample classes efficiently in order to achieve better classification performance. In this research paper, a new hybrid feature selection strategy for high-dimensional datasets is proposed to find the most discriminative subset of features for the dataset with the irrelevant and redundant features discarded. The proposed algorithm is called Maximal Clique based on the coefficients Ψ (MaCΨ algorithm). The MaCΨ method has the capability to handle categorical, numerical, and hybrid datasets. Furthermore, it can be applied either to binary or multi-class classification problems. The global structure of the MaCΨ algorithm can be described by three steps. In the first step, a weight is proposed to evaluate the importance of each feature in the dataset by balancing the trade-off between two novel measures of relevance and redundancy, and then the K most important features are selected to form the candidate subset, where K is taken as user input. In the second phase, a wrapper method based on graph theory is applied to the subset retained from the first step to extract the optimal subset of features. In the last stage, the final subset of features with the highest classification performance and the lowest number of features is obtained by applying the backward elimination algorithm to the optimal subset. The performance of the MaCΨ methodology is investigated on artificial as well as real-world datasets with different dimensionalities. The statistical analysis of the experimental results clearly indicates that the MaCΨ approach achieves competitive results in terms of the classification accuracy and the number of selected features compared with some state-of-the-art approaches. •Two novel measures are defined to evaluate the relevance and redundancy of each predictor of any type.•A new hybrid filter–wrapper feature selection approach is proposed to select the most important features.•The filter phase is based on a novel feature evaluation criterion (MaCΨ weight) that is related to the defined relevance and redundancy measures simultaneously.•The wrapper phase is based on graph theory and also on seq