Application of complex systems topologies in artificial neural networks optimization: An overview

•Overview of complex systems structures and topologies.•Review on application of small-world topology to neural networks.•Review on application of scale-free topology to neural networks.•Comparative analysis of neural network optimization methods. Through the success of artificial neural networks (ANNs) in different domains, intense research has been recently centered on changing the networks architecture to optimize the performance. Due to the broad connectivity and complex structure of artificial neural networks (ANNs), designing diluted ANNs with less time, wiring costs and space while obtaining high performance have attracted much attention. Complex systems theory, in which the influence of structure on the overall behavior of the network is mainly considered, have been applied in ANNs to have more efficient and less complex structures. It has been shown that complex random topologies outperform the fully-connected ANNs with less connectivity. But according to neurobiological investigations, highly clustered neurons with short characteristic path length and scale-free distributions are more favored while spending less connectivity costs. Therefore, applying small-world and scale-free topologies on ANNs has been explored in recent years. In this paper, the methodology and results of recent studies are summarized and discussed in which the authors investigated the efficacy of small-world, scale-free and hybrid complex networks on the performance of Hopfield associative memory and layered ANNs compared with conventional fully-connected and random structures.