Information Structures for Feedback Capacity of Channels With Memory and Transmission Cost: Stochastic Optimal Control and Variational Equalities

The Finite Transmission Feedback Information (FTFI) capacity is characterized for any class of channel conditional distributions {\mathbf{P}}_{B_{i}|B^{i-1}, A_{i}} and {\mathbf{P}}_{B_{i}|B_{i-M}^{i-1}, A_{i}} , where M is the memory of the channel, B^{i} \stackrel {\triangle }{=}\{B^{-1},B_{0}, \ldots, B_{i}\} are the channel outputs, and A^{i} \stackrel {\triangle }{=}\{A_{0}, A_{1}, \ldots, A_{i}\} , are the channel inputs, for i=0, \ldots, n . The characterizations of FTFI capacity are obtained by first identifying the information structures of the optimal channel input conditional distributions {\mathscr P}_{[0, n]} \stackrel {\triangle }{=}\big \{ {\mathbf{P}}_{A_{i}|A^{i-1}, B^{i-1}}: i=0, \ldots, n\big \} , which maximize directed information C_{A^{n} \rightarrow B^{n}}^{FB} \stackrel {\triangle }{=}\sup _{\mathscr P_{[0, n]} } I(A^{n} \rightarrow B^{n}), \hspace {.2in} I(A^{n} \rightarrow B^{n}) \stackrel {\triangle }{=}\sum _{i=0}^{n} I(A^{i};B_{i}|B^{i-1}) . The main theorem states that, for any channel with memory M , the optimal channel input conditional distributions occur in the subset satisfying conditional independence \stackrel {\circ }{\mathscr P}_{[0, n]} \stackrel {\triangle }{=}\big \{ {\mathbf{P}}_{A_{i}|A^{i-1}, B^{i-1}}= {\mathbf{P}}_{A_{i}|B_{i-M}^{i-1}}: i=0, \ldots, n\big \} , and the characterization of FTFI capacity is given by C_{A^{n} \rightarrow B^{n}}^{FB, M} \stackrel {\triangle }{=}\sup _{ \stackrel {\circ }{\mathscr P}_{[0, n]} } \sum _{i=0}^{n} I(A_{i}; B_{i}|B_{i-M}^{i-1}) . Similar conclusions are derived for problems with average cost constraints of the form \frac {1}{n+1} {\mathbf{E}}\Big \{