Research on real time feature extraction method for complex manufacturing big data

Big data related to manufacturing applications has the traits such as great quantity, multi-sources, low value density, high complexity, and dynamic state. Traditional feature extraction methods are incapable of meeting real-time demands. Therefore, a robust incremental on-line feature extraction method based on PCA (Principal Component Analysis), RIPCA (Robust Incremental Principal Component Analysis), is proposed. RIPCA adopts a sliding window to update new coming data stream and to filter outliers. The proposed method could ensure the accuracy of data analysis and meet real-time demands of big data processing for manufacturing applications. A test data set based on a semiconductor manufacturing process containing 1567 records with 590 features is used to demonstrate the availability of the proposed method. Experimental results show that the method can effectively extract features of the data stream in real time with high accuracy.