Adaptive Stepsize Rule for Consensus Optimization by Supervisory Control Architecture

We propose an adaptive stepsize rule for multi-agent based consensus optimization, to overcome drawbacks of the conventional diminishing stepsize rules. The proposed stepsize rule is based on an agreement degree of agent-wise gradients. It can archive both of fast approach and convergence at the early and last stages of iteration, respectively. Since the stepsize of each iteration is computed using a part of the global information of agents, a supervisory control architecture is required. We prove that the sequence generated by the consensus optimization algorithm using the proposed supervisory stepsize rule converges to the optimum solution. Moreover, we experimentally show that the proposed rule has better performance than the diminishing and constant stepsize rules.