ON THE FAMILIES OF SOLUTIONS TO GENERALIZED MAXIMUM ENTROPY AND MINIMUM CROSS-ENTROPY PROBLEMS

In a recent monograph, 3 the Entropy Maximization Postulate (EMP) and the associated Generalized Maximum Entropy Principle (GMEP) have been enunciated. The EMP states that when a probabilistic system is embedded within an information-theoretic framework, the information-theoretic entropy is always a maximum. Furthermore, it exerts a controlling influence on all the probabilistic entities in order to ensure their most unbiased states. The GMEP deals with the principles underlying the mutual interaction of the four probabilistic entities, namely: (1) entropy measure; (2) a set of linear moment constraints; (3) the a posteriori probability distribution; and (4) the a priori probability distribution. First, we explain the exact sense in which the GMEP constitutes a generalization over the well-known Jaynes' 1 Maximum Entropy Principle (MEP) and Kullback's 4 Minimum Cross-Entropy Principle (MCEP). The generalizations of the latter two principles are referred to as direct principles where the focus is on the determination of the a posteriori probability distributions. The GMEP spells out methodologies for the determination of any one probabilistic entity when the rest of the three are specified. Thus, in addition to the direct principles, we can also identify several Inverse Principles. This paper deals with problems associated with the existence and uniqueness of solutions to both direct and inverse principles. Results on uniqueness are staled in the form of four theorems which also serve to define the confines of applications of the GMEP to systems problems. The considerations that enter in determining existence are enumerated and illustrated by examples.