Translating Braille into Chinese based on improved CBHG model

Transforming Mandarin Braille to Chinese text is a significant but less focused machine translation task. CBHG is a building block used in the Tacotron text-to-speech model. Since Mandarin Braille is constructed from the pronunciation of Chinese characters, CBHG can be used to perform Braille–Chinese translation. Unfortunately, only relying on the convolution blocks in CBHG cannot effectively extract the features of Braille sequences. Two ways are proposed to improve the CBHG model: CBHG-SE and CBHG-ECA. The two modules adaptively recalibrate channel-wise feature responses by explicitly modeling interdependencies between channels in CBHG. The quality of representations produced by the network can also be improved. Meanwhile, the network can learn to use global information to emphasize informative features and suppress less useful ones selectively. CBHG-ECA has stronger feature recalibration capabilities than CBHG-SE due to its more direct correspondence between channels and their weights. These two models can achieve 92.23 BLEU and 91.48 BLEU on the Braille–Chinese dataset, outperforming CBHG and other neural machine translation models. •It is the first time to use NMT to implement Braille–Chinese translation in one step.•SEnet enables the CBHG model to adaptively recalibrate the channel-wise feature.•CBHG model with ECAnet can capture the local cross-channel information interaction.