Technical Note--An Expectation-Maximization Algorithm to Estimate the Parameters of the Markov Chain Choice Mode

We develop an expectation-maximization algorithm to estimate the parameters of the Markov chain choice model. In this choice model, a customer arrives into the system to purchase a certain product. If this product is available for purchase, then the customer purchases it. Otherwise, the customer transitions between the products according to a transition probability matrix until she reaches an available one and purchases this product. The parameters of the Markov chain choice model are the probability that the customer arrives into the system to purchase each one of the products and the entries of the transition probability matrix. In our expectation-maximization algorithm, we treat the path that a customer follows in the Markov chain as the missing piece of the data. Conditional on the final purchase decision of a customer, we show how to compute the probability that the customer arrives into the system to purchase a certain product and the expected number of times that the customer transitions from a certain product to another one. These results allow us to execute the expectation step of our algorithm. Also, we show how to solve the optimization problem that appears in the maximization step of our algorithm. Our computational experiments show that the Markov chain choice model, coupled with our expectation-maximization algorithm, can yield better predictions of customer choice behavior when compared with other commonly used alternatives.