Understanding species richness-productivity relationships: the importance of meta-analyses

In the above quote, Levins was referring to "truth" and "lies" in model building, however, I believe his comments are relevant to the analysis of empirical data as well. We all recognize that published papers differ in quality, even those that are predominantly descriptive. Whittaker (2010), in his critique of meta-analyses of species richness productivity relationship (SRPRs), argues that few of the studies used in past meta-analyses of SRPRs are fit for the purpose. This leads him to "call time" on any further meta-analyses of SRPRs and to denounce the findings of previous meta-analyses as unreliable. Whittaker (2010:2524) states, "If the data aren't appropriate to meta-analysis, it is invalid to proceed with one. The solution [my italics] is to read the literature, think about it, and do one of the following: (1) devise some critical experimental or other rigorous field study that will make a meaningful contribution to the question to hand, (2) undertake a narrative review, or (3) carry out what Slavin (1995) has termed 'best evidence synthesis."' I would argue, however, that these alternatives do not provide a solution and that the past meta-analyses of SRPRs, despite their weaknesses and disagreements, have significantly advanced our understanding of these relationships. The literature on SRPRs is uneven in quality and heterogeneous in method: on that there is no doubt. But, it is what we have to work with. In the end, we make progress by scrutinizing our ideas in light of the available data. At the risk of sounding extreme, I suggest that even empiricists must look for "truth" at the intersection of independent "lies." Consider what we "knew" about SRPRs prior to the published meta-analyses.