Bridging Feature Selection and Extraction: Compound Feature Generation

Dimensionality reduction is an essential pre-processing technique in many of the data analysis tasks. Popular approaches for dimensionality reduction are Feature Selection (FS) and Feature Extraction (FE). Till now, these approaches are often studied separately or independently so that the final result contains either original or transformed features. In our work, we propose to bridge these two approaches with the aim of finding reduced feature set to contain both kinds (original as well as transformed) of features. A new framework, called Minimum Projection error Minimum Redundancy (MPeMR), is introduced to obtain this result while maintaining orthogonality property among selected original and linear combinations of features. A unified iterative algorithm, for both supervised and unsupervised cases, is also developed under this framework. For each case, the performance of the proposed algorithm is successfully compared with the state-of-the-art methods on real-life data sets.