Leveraging deep feature learning for wearable sensors based handwritten character recognition

Despite rapid advancements in technology, handwritten characters still hold significant roles in various fields, including education, communication, biometric signature verification, and health care. These applications often require digitization of the handwritten characters and associated hand movements to facilitate effective analysis and interpretation of the underlying task. Offline and online handwriting recognition are crucial steps involving the digitization of handwritten characters. Most of these existing systems actively use image processing techniques that are highly sensitive to environmental lighting conditions. Surface Electromyography signals (sEMG), being invariant to lighting conditions, are used in online handwriting recognition to facilitate the automatic transcription of handwritten characters. In this article, we have leveraged deep representation learning to build an efficient and robust sEMG-based Handwritten Character Recognition (HCR) pipeline. A Stacked sparse denoising autoencoder network is applied to obtain an effective deep feature representation. These rich low dimensional features obtained are further introduced into basic classifiers, producing state-of-the-art accuracy for the task. Additional experiments were performed to analyze the effect of the fusion of complementary sensing modules (Accelerometer and Gyroscope) on the performance of sEMG based HCR pipeline. Extensive evaluations were performed to ensure the validity of the obtained results. For the experimentation, new datasets consisting of sEMG, Accelerometer, and Gyroscope signals corresponding to 26 handwritten lower English alphabets were collected from 15 subjects. Our proposed pipeline can be used to build real-time Human–Computer Interaction(HCI) applications for smart classrooms facilitating digitization of handwritten notes and clinical applications involving handwriting analysis tasks. •We explored autoencoders for handwritten characters recognition using sEMG signals.•We recorded sEMG signals of 15 subjects while writing 26 lowercase English alphabets.•Deep stacked sparse denoising autoencoders were applied on this new dataset.•Further experimental analysis was done by coupling sEMG with IMU sensor module.•Our proposed approach achieved an accuracy of 98.72% even with baseline classifiers.•Our model can potentially be used in smart-classrooms and clinical applications.