What is wrong with the meta-analyses on honey and oral mucositis due to cancer therapies?

•The work shows that meta-analyses on oral mucositis have severe flaws.•The flaws in most reviews have not become apparent and have not been discussed.•Peer review processes can not always insure that meta-analyses are properly done. The results of meta-analyses currently represent the highest level of evidence in modern medicine. Taking the example of radiotherapy and radiochemotherapy-induced oral mucositis and the effects of honey, we analysed six meta-analyses on the topic to assess the quality of the meta-analyses. We analysed the various meta-analyses in detail and compared whether the authors have correctly included the various trials or not. We found that the quality of these meta-analyses was low. Especially the more recent meta-analyses included trials in which radiotherapy was not part of the medical intervention or where substances other than pure honey were used. It is impossible to determine the underlying reasons why these meta-analyses were able to pass the peer-review system without the request for adequate improvements prior to publication. According to the literature at least 7% of the included meta-analyses revealed false results, but it was assumed that due to limitations of external validity and to the decreased likelihood of updating positive meta-analyses, the true proportion of false positives in the meta-analysis was likely to be higher. However, it is crucial that when severe flaws in meta-analyses are detected that they be reported or the meta-analyses are withdrawn, otherwise the normal reader will take the results as given.