Evaluation of methodological search filters—a review

Objectives: The objectives of the study were: to provide an overview of approaches to methodological search filter development; to identify and critically review the stages of methodological search filter development; to devise a search filter appraisal checklist based on the review. Methods: An iterative approach to searching was employed utilizing health and library databases, the world wide web and citation searching. Further systematic methods included hand searching of key journals in the field of search filter development, contacting known experts in the field and scanning reference lists of relevant papers to identify additional studies. Altogether, 51 potentially relevant papers were found, of which 20 met the inclusion criteria. Results: Four stages of search filter development were identified from the literature (search term selection, identification of a gold standard, evaluation and validation). Variations in the methods used to approach these four stages were identified, most importantly in the extent to which search filters are tested and validated. Conclusion: Awareness of the process and limitations involved in search filter development is essential to make an informed decision on the applicability and validity of search filters. The findings of this review indicate a considerable agenda for future research, in particular, to improve the quality of reporting of search filters and to inform users on their use and application. Based on the review, guidance in the appraisal process of search filters is given in the form of a checklist. Key Messages Implications for Policy • Improvements are needed in the quality of reporting of search filters • Further research is warranted investigating the methods used to develop search filters • Evaluation and impact of search filters on end‐user searching outcome is essential to establish the effectiveness of search filters Implications for Practice • Appraisal of search filters for applicability and validity is required • Clearer guidance for end‐users (librarians, clinicians) on the use of search filters is needed