Deep imitator: Handwriting calligraphy imitation via deep attention networks

•Handwriting style features are extracted by a Convolution Neural Network.•Calligraphy embedding is computed by attention and orthogonal mata-style matrix.•Calligraphy and character embedding are the inputs of Dual-condition Gated Recurrent Unit.•Handwriting Imitation results are generated by Gaussian Mixture Model and sampling.•Subjective and quantitative tests verify the imitation performance. Calligraphy imitation (CI) from a handful of target handwriting samples is such a challenging task that most of the existing writing style analysis or handwriting generation methods do not exhibit satisfactory performance. In this paper, we propose a novel multi-module framework to address the problem of CI. Firstly, we utilized a deep convolution neural network (CNN) to extract personalized calligraphical features. Then we built a calligraphy-clustering attention module and a mata-style matrix (msM) to compute an embedding of calligraphy. The structure of conditional gated recurrent unit (cGRU) is then improved to predict the probabilistic density of pen tip movement displacement by dual condition inputs. Finally, we generated personalized handwriting stroke sequences through iterative sampling with Gaussian mixture model (GMM). Experiments on public online handwriting databases verify that the proposed method could achieve satisfactory performance; the generated samples achieved high similarities with original handwriting examples.