A survey of fractional calculus applications in artificial neural networks

Artificial neural network (ANN) is the backbone of machine learning, specifically deep learning. The interpolating and learning ability of an ANN makes it an ideal tool for modelling, control and various other complex tasks. Fractional calculus (FC) involving derivatives and integrals of arbitrary non-integer order has recently been popular for its capability to model memory-type systems. There have been many attempts to explore the possibilities of combining these two fields, the most popular combination being the use of fractional derivative in the learning algorithm. This paper reviews the use of fractional calculus in various artificial neural network architectures, such as radial basis functions, recurrent neural networks, backpropagation NNs, and convolutional neural networks. These ANNs are popularly known as fractional-order artificial neural networks (FANNs). A detailed review of the various concepts related to FANNs, including activation functions, training algorithms based on fractional derivative, stability, synchronization, hardware implementations of FANNs, and real-world applications of FANNs, is presented. The study also highlights the advantage of combining fractional derivatives with ANN, the impact of fractional derivative order on performance indices like mean square error, the time required for training and testing FANN, stability, and synchronization in FANN. The survey reports interesting observations: combining FC to an ANN endows it with the memory feature; Caputo definition of fractional derivative is the most commonly used in FANNs; fractional derivative-based activation functions in ANN provide additional adjustable hyperparameters to the networks; the FANN has more degree of freedom for adjusting parameters compared to an ordinary ANN; use of multiple types of activation functions can be employed in FANN, and many more.