Accuracy and Architecture Studies of Residual Neural Network Method for Ordinary Differential Equations

In this paper, we investigate residual neural network (ResNet) method to solve ordinary differential equations. We verify the accuracy order of ResNet ODE solver matches the accuracy order of the data. Forward Euler, Runge–Kutta2 and Runge–Kutta4 finite difference schemes are adapted generating three learning data sets, which are applied to train three ResNet ODE solvers independently. The well trained ResNet solvers obtain 2nd, 3rd and 5th orders of one step errors and behave just as its counterpart finite difference method for linear and nonlinear ODEs with regular solutions. In particular, we carry out (1) architecture study in terms of number of hidden layers and neurons per layer to obtain optimal network structure; (2) target study to verify the ResNet solver is as accurate as its finite difference method counterpart; (3) solution trajectory simulations. A sequence of numerical examples are presented to demonstrate the accuracy and capability of ResNet solver.