Neural machine translation with Gumbel Tree-LSTM based encoder

•An unsupervised tree-to-sequence neural machine translation method is proposed.•Learning latent trees for improving neural machine translation.•Comparisons between Gumbel Tree-based model and the baselines, RNMT and Transformer.•Analysis on the learned tree structures and attention scores on the structures. Neural machine translation has improved the translation accuracy greatly and received great attention of the machine translation community. Tree-based translation models aim to model the syntactic or semantic relation among long-distance words or phrases in a sentence. However, it faces the difficulties of expensive manual annotation cost and poor automatic annotation accuracy. In this paper, we focus on how to encode a source sentence into a vector in a unsupervised-tree way and then decode it into a target sentence. Our model incorporates Gumbel Tree-LSTM, which can learn how to compose tree structures from plain text without any tree annotation. We evaluate the proposed model on both spoken and news corpora, and show that the performance of our proposed model outperforms the attentional seq2seq model and the Transformer base model.