Optimal model for a Markov chain with Markov covariates

In this paper, we develop a model selection procedure for paired stochastic processes (Yt, Xt) where the Markov chain Xt chooses the set of transition probabilities to be used by Yt at time t. More precisely Xt is an order p Markov chain and given the value of Xt at time t, the (response) process Yt is an order o Markov chain with transition probabilities that depend on the specific value of Xt. This situation is different from Hidden Markov models and Double Markov chains (see [1]) since in our case, both processes Xt and Yt are observed. We define a family of models for this situation and investigate a consistent model selection procedure such that the final model has a minimal number of parameters. The methodology used to select a model is based on the Bayesian Information Criterion (BIC) by Schwarz [2]. The procedure consists of finding a partition of the joint state space determining the relationship between the joint processes of covariate and the response. The model and the model selection methodology, itroduced here, are generalizations of the Partition Markov models by Garćıa and González-López [3], [4], [5], and [6].