Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping

This work proposes block-coordinate fixed point algorithms with applications to nonlinear analysis and optimization in Hilbert spaces. The asymptotic analysis relies on a notion of stochastic quasi-Fejér monotonicity, which is thoroughly investigated. The iterative methods under consideration feature random sweeping rules to select arbitrarily the blocks of variables that are activated over the course of the iterations and they allow for stochastic errors in the evaluation of the operators. Algorithms using quasi-nonexpansive operators or compositions of averaged nonexpansive operators are constructed, and weak and strong convergence results are established for the sequences they generate. As a by-product, novel block-coordinate operator splitting methods are obtained for solving structured monotone inclusion and convex minimization problems. In particular, the proposed framework leads to random block-coordinate versions of the Douglas–Rachford and forward-backward algorithms and of some of their variants. In the standard case of m = 1 block, our results remain new as they incorporate stochastic perturbations. 1. Introduction. The main advantage of block-coordinate algorithms is to result in implementations with reduced complexity and memory requirements per iteration. These benefits have long been recognized [3, 18, 50] and have become increasingly important in very-large-scale problems. In addition, block-coordinate strategies may lead to faster [20] or distributed [41] implementations. In this paper, we propose a block-coordinate fixed point algorithmic framework to solve a variety of problems in Hilbertian nonlinear numerical analysis and optimization. Algorithmic fixed point theory in Hilbert spaces provides a unifying and powerful framework for the analysis and the construction of a wide array of solution methods in such problems [5, 7, 19, 22, 66]. Although several block-coordinate algorithms exist for solving specific optimization problems in Euclidean spaces, a framework for dealing with general fixed point methods in Hilbert spaces and which guarantees the convergence of the iterates does not seem to exist at present. In the proposed constructs, a random sweeping strategy is employed for selecting the blocks of coordinates which are activated over the iterations. The sweeping rule allows for an arbitrary sampling of the indices of the coordinates. Furthermore, the algorithms tolerate stochastic errors in the implementation of the operators. This