Designs to Improve Capability of Neural Networks to Make Structural Predictions

A deep neural network model generally consists of different modules that play essential roles in performing a task. The optimal design of a module for use in modeling a physical problem is directly related to the success of the model. In this work, the effectiveness of a number of special modules, the self-attention mechanism for recognizing the importance of molecular sequence information in a polymer, as well as the big-stride representation and conditional random field for enhancing the network ability to produce desired local configurations, is numerically studied. Network models containing these modules are trained by using the well documented data of the native structures of the HP model and assessed according to their capability in making structural predictions of unseen data. The specific network design of self-attention mechanism adopted here is modified from a similar idea in natural language recognition. The big-stride representation module introduced in this work is shown to drastically improve network’s capability to model polymer segments of strong lattice position correlations.