Sentiment analysis with word-based Urdu speech recognition

Urdu is one of the popular languages across the world as approximately 70 million people speak Urdu in their day-to-day conversations. In general, Muslims prefer to share their opinion or feedback in speech format in the Urdu language. From the literature, it is evident that opinion extraction from naturalistic audio has emerged as a new field of research. In this automatic speech, recognition is carried with keyword spotting approaches on audio, and then opinion score is computed. In this paper, the authors propose a novel framework for the extraction of sentiment from Urdu audio data. Firstly, speech utterances are duly pre-processed, and then short-term features such as Mel-frequency cepstral coefficients, spectral energy, Chroma vector features, perceptual linear prediction (PLP) cepstral coefficients and relative-spectral PLP features are extracted. Five mid-term features, including mean, median, etc., are then derived from those short-term features. In the opinion extraction phase, midterm features of Urdu test utterances are compared with the midterm features of the dictionary of words to cite the opinion as positive, negative, and neutral. The originality of the work involves analyzing the perceptual features to find out the features that contain significant information to extract sentiment in Urdu utterances. In this work, weight mean vector fusion technique is used to fuse the outputs of hidden Markov model and dynamic time warping. In the experiments, 97.1% accuracy is achieved in the sentiment analysis task on the Urdu custom corpus of 600 utterances, which outperforms other state-of-the-art classifiers.