Exit Frequency Matrices for Finite Markov Chains
Consider a finite irreducible Markov chain on state space S with transition matrix M and stationary distribution π. Let R be the diagonal matrix of return times, Rii = 1/πi. Given distributions σ, τ and k ∈ S, the exit frequency xk(σ, τ) denotes the expected number of times a random walk exits state...
Gespeichert in:
Veröffentlicht in: | Combinatorics, probability & computing probability & computing, 2010-07, Vol.19 (4), p.541-560 |
---|---|
Hauptverfasser: | , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Consider a finite irreducible Markov chain on state space S with transition matrix M and stationary distribution π. Let R be the diagonal matrix of return times, Rii = 1/πi. Given distributions σ, τ and k ∈ S, the exit frequency xk(σ, τ) denotes the expected number of times a random walk exits state k before an optimal stopping rule from σ to τ halts the walk. For a target distribution τ, we define Xτ as the n × n matrix given by (Xτ)ij = xj(i, τ), where i also denotes the singleton distribution on state i. The dual Markov chain with transition matrix = RM⊤R−1 is called the reverse chain. We prove that Markov chain duality extends to matrices of exit frequencies. Specifically, for each target distribution τ, we associate a unique dual distribution τ*. Let $\rX_{\fc{\t}}$ denote the matrix of exit frequencies from singletons to τ* on the reverse chain. We show that $\rX_{\fc{\t}} = R (X_{\t}^{\top} - \vb^{\top} \one)R^{-1}$, where b is a non-negative constant vector (depending on τ). We explore this exit frequency duality and further illuminate the relationship between stopping rules on the original chain and reverse chain. |
---|---|
ISSN: | 0963-5483 1469-2163 |
DOI: | 10.1017/S0963548310000118 |