Interpretable Deep Learning for Nonlinear System Identification Using Frequency Response Functions With Ensemble Uncertainty Quantification

Deep learning methods contain powerful tools for modelling nonlinear dynamic systems. However, whilst these models are useful for predicting outputs, they tend to be described by complicated black box equations that lack interpretability. They are therefore not so useful for giving insight into system dynamics, and importantly, insight into why a system produces a certain output in response to a given input. This paper presents a novel method for interpreting and comparing deep learning models for nonlinear system identification, using nonlinear output frequency response functions (NOFRFs). NOFRFs describe nonlinear dynamic system behaviour in the frequency-domain using one-dimensional functions, in a manner similar to how Bode plots are used for analysing linear dynamic systems. This is a classical way of interpreting and understanding system behaviour, e.g. via resonances, and in the case of nonlinear systems, super and sub-harmonics, and energy transfer between frequencies. We also use uncertainty quantification via an ensemble bootstrap method to enhance the model interpretation, by propagating the model uncertainty estimates into the frequency-domain. The approach is demonstrated with gated recurrent unit (GRU) and long short term memory (LSTM) models - both are types of recurrent network used in deep learning that are analogous to nonlinear state space models. The results obtained from both a numerical example (a nonlinear mass spring damper system that exhibits energy transfer between frequencies) and a real-world nonlinear system (a magneto-rheological damper) show that it is possible to gain valuable insight and interpretation of the system dynamics from the NOFRFs in a way that is not possible from analysing the time-domain model equations alone.