Systematic Reviews & Meta-Analysis

A systematic review attempts to identify, appraise and synthesize all available relevant evidence that to answer a specific, focus research question. Researchers conducting systematic reviews use standardized, systematic methods and pre-selected eligibility criteria to reduce the risk of bias in identifying, selecting and analyzing relevant studies.

Prepared by the Cochrane Consumers and Communication Group, La Trobe University and generously support by Cochrane Australia. Written by Jack Nunn and Sophie Hill.

Traditional literature reviews differ from systematic reviews in many ways, and especially on how they are conducted and time commitment required. 

When  systematic reviews SHOULD be done:

• If you have a clearly defined research question with established inclusion and exclusion criteria
• To test a specific hypothesis to ensure a manageable results set
• When there is a large body of primary research on your specific research question
• When you have team of at least three people assembled to help conduct the systematic review
• When a transparent search methodology and replicability are needed
• When an existing systematic review is outdated (consider updating the existing review)
• When no ongoing or existing systematic review addresses your research question

When Systematic reviews SHOULD NOT be done:

• If you do not have a clearly defined research question.
  • Systematic reviews without a clear and specified research question with details such as populations, interventions, exposures, and outcomes, will produce large and inconsistent search results to screen, and offer no consistent way to assess and synthesize findings from the studies that are identified.
• If you do not have enough time to complete the review systematically
  • Systematic reviews are a lot of work. Including creating the protocol, building and running a quality search, collecting all the papers, evaluating the studies that meet the inclusion criteria and extracting and analyzing the summary data, a well done review can require dozens to hundreds of hours of work that can span several months.
• If you plan to do a systematic review on your own
  • All systematic review guidelines recommend that at least two subject experts screen the studies identified in the search. The first round of screening can consume 1 hour per screener for every 100-200 records. A systematic review is a team effort.

Systematic reviews require time and effort to complete. It should not be expected to be complete a systematic review in a matter of months. An average time to complete a systematic review is between 12-18 months. The Cochrane Handbook for Systematic Reviews of Interventions suggests the following timeline to complete a review:

Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from https://training.cochrane.org/handbook/current.

A systematic review can't be done alone. You should carefully consider all of the expertise you will need to define your research question, search for evidence, appraise/grade the evidence, and potentially complete a statistical meta-analysis of the data. A recommended systematic review team would consist of the following:

• 2 or more subject experts on the topic of the study. These experts will screen and appraised the evidence, and a third may be necessary to settle any disagreements.
• A library or expert skillful enough to perform complex literature search.
• A statistician if a meta-analysis is to be performed.