Optimized Hybrid Convolution Neural Network with Machine Learning for Arabic Sign Language Recognition

The World Health Organization stated that the global population of hard of hearing individuals is estimated to exceed 360 million people, and this number is continuously increasing. Communication barriers between these individuals and hearing individuals pose significant challenges in many areas of life, including education and employment. Therefore, developing methods to facilitate communication and bridge communication gaps is essential. In this paper, a novel approach is presented to Arabic sign alphabets recognition using optimized hybrid techniques. The proposed approach combines a convolutional neural network with five traditional machine learning algorithms: Feedforword Neural network, Decision Tree, Random Forest, Support Vector Machine and K-Nearest Neighbors. The proposed approach was evaluated on a large dataset called ArSL2018, which consists of 32 different classes of Arabic alphabet letters. Six different optimization techniques were investigated to find the optimal multipliers for the outputs of the Hybrid CNN models to achieve high classification accuracy. These techniques included the Genetic Algorithm, Particle Swarm, Firefly Algorithm, Differential Evolution, Sine Cosine, and Harris Hawks Optimization. The experiments demonstrated that the optimized hybrid techniques achieved the highest accuracy, approaching 99%, surpassing their counterparts. This demonstrates the efficiency of the proposed model in accurately recognizing Arabic alphabets.