Multimodal biometric identification: leveraging convolutional neural network (CNN) architectures and fusion techniques with fingerprint and finger vein data

Advancements in multimodal biometrics, which integrate multiple biometric traits, promise to enhance the accuracy and robustness of identification systems. This study focuses on improving multimodal biometric identification by using fingerprint and finger vein images as the primary traits. We utilized the "NUPT-FPV" dataset, which contains a substantial number of finger vein and fingerprint images, which significantly aided our research. Convolutional neural networks (CNNs), renowned for their efficacy in computer vision tasks, are used in our model to extract distinct discriminative features. Specifically, we incorporate three popular CNN architectures: ResNet, VGGNet, and DenseNet. We explore three fusion strategies used in security applications: early fusion, late fusion, and score-level fusion. Early fusion integrates raw images at the input layer of a single CNN, combining information at the initial stages. Late fusion, in contrast, merges features after individual learning from each CNN model. Score-level fusion employs weighted aggregation to combine scores from each modality, leveraging the complementary information they provide. We also use contrast limited adaptive histogram equalization (CLAHE) to enhance fingerprint contrast and vein pattern features, improving feature visibility and extraction. Our evaluation metrics include accuracy, equal error rate (EER), and ROC curves. The fusion of CNN architectures and enhancement methods shows promising performance in identifying multimodal biometrics, aiming to increase identification accuracy. The proposed model offers a reliable authentication system using multiple biometrics to verify identity.