Using Parallel Computation to Improve Independent Metropolis-Hastings Based Estimation

In this article, we consider the implications of the fact that parallel raw-power can be exploited by a generic Metropolis-Hastings algorithm if the proposed values are independent from the current value of the Markov chain. In particular, we present improvements to the independent Metropolis-Hastings algorithm that significantly decrease the variance of any estimator derived from the MCMC output, at a null computing cost since those improvements are based on a fixed number of target density evaluations that can be produced in parallel. The techniques developed in this article do not jeopardize the Markovian convergence properties of the algorithm, since they are based on the Rao-Blackwell principles of Gelfand and Smith (1990), already exploited in the work of Casella and Robert (1996), Atchadé and Perron (2005), and Douc and Robert (2011). We illustrate those improvements both on a toy normal example and on a classical probit regression model, but stress the fact that they are applicable in any case where the independent Metropolis-Hastings is applicable. The code used in this article is available as supplementary material.