Selection-mutation dynamics of signaling games

We study the structure of the rest points of signaling games and their dynamic behavior under selection-mutation dynamics by taking the case of three signals as our canonical example. Many rest points of the replicator dynamics of signaling games are not isolated and, therefore, not robust under perturbations. However, some of them attract open sets of initial conditions. We prove the existence of certain rest points of the selection-mutation dynamics close to Nash equilibria of the signaling game and show that all but the perturbed rest points close to strict Nash equilibria are dynamically unstable. This is an important result for the evolution of signaling behavior, since it shows that the second-order forces that are governed by mutation can increase the chances of successful signaling.