A comparative study of preprocessing mismatch effects in color image based face recognition

Face color information can play an important role in face recognition (FR) and it can be used to considerably improve FR performance obtained using only grayscale images. The color-based FR methods involve a preprocessing step where a color image is converted into either a monochromatic image or an image having a different color representation. In practical FR systems, the recording or transmission format of the testing images may be arbitrary or inconsistent depending on the application (e.g., face images could consist of grayscale or color pixels either in compressed or uncompressed form). Further, a wide variety of grayscale and color conversions can be used in the preprocessing step. This could lead to a so-called preprocessing mismatch in color-based FR methods: the training and testing face images, generated after preprocessing, do not match in terms of their degree of compression or in terms of their grayscale or color representations. In contrast to grayscale-based FR, a practical color-based FR system has a higher chance of being confronted with a preprocessing mismatch. The aim of this paper is to present a comparative study that addresses the impact of a preprocessing mismatch on color-image based FR methods. We explore three different types of preprocessing mismatches, which practical color-based FR system are highly likely to encounter. In addition, comparative and extensive experiments have been carried out to analyze the effects of the preprocessing mismatches on an FR performance, using Color FRETET, CMU-PIE, AR, and SCface public face databases. The important conclusions drawn from our experiments include: (1) of all color-based FR methods under consideration, color-based FR using feature-level fusion is the most robust approach to preprocessing mismatches; (2) the preprocessing mismatch caused by the use of compressed color images can significantly deteriorate FR performance of color-based FR methods; (3) grayscale testing images can be critical for the feasibility of color-based FR using an input-level fusion; (4) the preprocessing mismatch in terms of grayscale representation has little effect on the FR performances of color-based FR methods.