Effects of Directionality of Significance Tests on the Bias of Accessible Effect Sizes

The proportion of studies that use one-tailed statistical significance tests (π) in a population of studies targeted by a meta-analysis can affect the bias of the sample effect sizes (sample ESs, or d s) that are accessible to the meta-analyst. H. C. Kraemer, C. Gardner, J. O. Brooks, and J. A. Yesavage (1998) found that, assuming π = 1.0, for small studies (small N s) the overestimation bias was large for small population ESs (δ ≤ 0.2) and reached a maximum for the smallest population ES (viz., δ = 0). The present article shows (with a minor modification of H. C. Kraemer et al.'s model) that when π = 0, the small- N bias of accessible sample ESs is relatively small for δ ≤ 0.2, and a minimum (in fact, nonexistent) for δ = 0. Implications are discussed for interpretations of meta-analyses of (a) therapy efficacy and therapy effectiveness studies, (b) comparative outcome studies,and (c) studies targeting small but important population ESs.