Fusing nonlinear solvers with transformers for accelerating the solution of parametric transient problems

In the field of computational science and engineering, solving nonlinear transient problems still poses a challenging task that often requires significant computational resources. This research introduces a novel methodology that harnesses the power of cutting-edge Temporal Fusion Transformers (TFTs) to accelerate the solution of such problems in multi-query scenarios (i.e. parameterized problems). At each time step, TFT models, renowned for their time series forecasting capabilities, are combined with dimensionality reduction techniques to efficiently generate initial solutions for nonlinear solvers. Specifically, during the training phase, a reduced set of high-fidelity system solutions is obtained by solving the system of differential equations governing the problem for different parameter instances. Then, dimensionality reduction is applied to create a reduced latent space to simplify the representation of the complex system solutions. Subsequently, TFT models are trained for one-step-ahead forecasting in the latent space, utilizing information from previous states to make accurate predictions about future states. The TFTs’ predictions are fed back to the system as initial guesses at each time step of the solution algorithm and are then guided towards the exact solutions that satisfy equilibrium using Newton–Raphson (NR) iterations. The basic premise of the proposed idea is that having accurate initial predictions will significantly decrease the number of the costly NR-iterations needed in nonlinear dynamic problems, effectively reducing the solution time. In addition, the proposed scheme is able to handle problems with parametric and time-variant forcing forces. A customized TFT architecture is developed that takes as input not only the response history of the system, but also the current loading, and makes informed guesses about the future state of the system. The methodology’s effectiveness is demonstrated in numerical applications that involve high nonlinearity, where the TFT-generated initial solutions resulted in a notable reduction in the number of NR-iterations required for solver convergence. This significant enhancement in computational efficiency holds substantial promise, especially in scenarios involving a multitude of analyses and high iteration demands, with wide-ranging applications across computational mechanics and related fields. •Methodology to accelerate solving parametric nonlinear PDEs with time-varying terms.•Uses dimensionality