An approach for Malayalam sentence to Indian sign language generation using synthetic animations

An average of 360 million people worldwide suffer from hearing disabilities which are considered 5% of the total population. This number is expected to cross 900 million by 2050. The majority of auditory-disabled people live in low-income and middle-income countries. As per the 2011 census, approximately 7 million Indians suffer from hearing and speech impairment. People with severe or profound hearing loss are unable to listen to others and thus face problems in communication. This lack of communication can have a substantial impact on the personality of the deaf person, which may include loneliness, isolation, and frustration. Sign Language is one of the most important languages used by the Deaf community for their communication. In sign language, Deaf people express themselves through gestures and facial expressions. Every gesture represents a word or a phrase in the natural language. Sign languages are different in different regions, around 300 sign languages are used all over the world. Indian Sign Language (ISL) is the language used by the Deaf communities in India. Hearing people communicate through speech, and the hearing community does not understand sign language. So, direct communication between a hearing loss and a hearing person is difficult. So, a system that can translate text to sign language will help reduce the communication gap between a Deaf and a hearing person. We have developed an Indian Sign Language generation system in which synthetic animations are generated based on the inputted text. To enable both hearing and deaf people to understand the structure of Malayalam and ISL sentences, we have developed a Malayalam sentence to ISL video generation system. Currently, text-to-ISL translation systems in Indian languages focus on word-to-ISL translation, and sentence-to-ISL translation is done for English. The proposed Malayalam sentence to ISL translation system consists of mainly two steps, Malayalam sentence Gloss representation, and Gloss to video generation. Gloss is a Signwriting notation that helps represent the words in the form of symbols, and notations. In Malayalam sentence to Gloss conversion, mainly we have to focus on the grammar rules of both Malayalam and ISL. No standardized grammar rulesare available for Malayalam to ISL translation, like English to American Sign Language (ASL) or British Sign Language (BSL) translation. We have developed a Malayalam Sentence-Gloss database with the help of a Deaf who is expert in ISL