Meta‐analysis of magnitudes, differences and variation in evolutionary parameters

Meta‐analysis is increasingly used to synthesize major patterns in the large literatures within ecology and evolution. Meta‐analytic methods that do not account for the process of observing data, which we may refer to as ‘informal meta‐analyses’, may have undesirable properties. In some cases, informal meta‐analyses may produce results that are unbiased, but do not necessarily make the best possible use of available data. In other cases, unbiased statistical noise in individual reports in the literature can potentially be converted into severe systematic biases in informal meta‐analyses. I first present a general description of how failure to account for noise in individual inferences should be expected to lead to biases in some kinds of meta‐analysis. In particular, informal meta‐analyses of quantities that reflect the dispersion of parameters in nature, for example, the mean absolute value of a quantity, are likely to be generally highly misleading. I then re‐analyse three previously published informal meta‐analyses, where key inferences were of aspects of the dispersion of values in nature, for example, the mean absolute value of selection gradients. Major biological conclusions in each original informal meta‐analysis closely match those that could arise as artefacts due to statistical noise. I present alternative mixed‐model‐based analyses that are specifically tailored to each situation, but where all analyses may be implemented with widely available open‐source software. In each example meta‐re‐analysis, major conclusions change substantially.