Investigation on the high-order approximation of the entropy bias

The estimation of entropy from experimental data has a considerable bias when the discretization of the variable domain is comparable to the sample size. In this case, the source of the bias is the difference between the a priori distribution and the observed distribution from sampled data. In this paper, we estimate the entropy bias considering an infinite sum of central moments of the binomial distribution using two probability mass functions. We analyze the bias in the light of the ratio between the number of the partition of the domain and the sample size. The main motivation of this study is improving statistical hypothesis testing in which probabilities are conceived beforehand. We examine the adequacy of high-order approximation according to the ratio between the sample size and the number of domain partitions. Finally, we expand the analysis to the entropy-derived mutual information and present an application for network reconstruction. •Estimation of the entropy bias via a sum of moments of probability functions.•Adequacy of high-order expansion in information-theory functionals•Hypothesis testing using entropy-derived mutual information.•Application in network reconstruction.