Selecting feature subsets for inducing classifiers using a committee of heterogeneous methods

As a previous step to machine learning (ML) induced classifiers, attribute subset selection methods have become an efficient alternative for reducing the dimensionality of the search space, with obvious benefits to the learning techniques used. This paper investigates the problem of feature subset selection using a committee of filter, wrapper and embedded methods. The wrappers were implemented using two different search mechanisms, a genetic algorithm and a best-first procedure as well as three different machine learning paradigms: instance-based (nearest neighbor - NN), neural network (DistAl) and symbolic (C4.5). The two filter methods used are based on consistency and correlation measures. The goals of the experiments were to be able to identify the most suitable attribute subsets to be further used for inducing a classifier as well as investigate if the combination of different results given by the committee's members can outperform any machine learning method using the original training set.