Systematic Review | Definition, Example & Guide

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

What is a systematic review?

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias. The methods are repeatable, and the approach is formal and systematic:

1. Formulate a research question
2. Develop a protocol
3. Search for all relevant studies
4. Apply the selection criteria
5. Extract the data
6. Synthesize the data
7. Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative (qualitative), quantitative, or both.

Systematic review vs. meta-analysis

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis. A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size.

Receive feedback on language, structure and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Grammar
• Style consistency

See an example

Systematic review vs. literature review

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Systematic review vs. scoping review

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

When to conduct a systematic review

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention, such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question, usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.

• A team of at least three people. Best practices require three people for certain steps of the systematic review process. Ideally, in addition to your research team you’ll also have an advisory group of about six people.
  • If you’re doing a systematic review on your own (e.g., for a research paper or thesis), you should take appropriate measures to ensure the validity and reliability of your research.

• Access to databases and journal archives. Often, your educational institution provides you with access.

• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.

• Bibliographic, word-processing, spreadsheet, and statistical software. For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

Pros and cons of systematic reviews

A systematic review has many pros.

• They minimize bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent, so they can be scrutinized by others.
• They’re thorough: they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons.

• They’re time-consuming.
• They’re narrow in scope: they only answer the precise research question.

Step-by-step example of a systematic review

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO:

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)
• Outcome(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P?

Sometimes, you may want to include a fifth component, the type of study design. In this case, the acronym is PICOT.

• Type of study design(s)

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information: Provide the context of the research question, including why it’s important.
• Research objective(s): Rephrase your research question as an objective.
• Proposed methods
  • Selection criteria: State how you’ll decide which studies to include or exclude from your review.
  • Search strategy: Discuss your plan for finding studies.
  • Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee. This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov.

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus. Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.

• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.

• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD). In medicine, clinical trial registries are another important type of gray literature.

• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator.

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol. The third person’s job is to break any ties.

To increase inter-rater reliability, ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

1. Based on the titles and abstracts: Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
2. Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

1. Information about the study’s methods and results. The exact information will depend on your research question, but it might include the year, study design, sample size, context, research findings, and conclusions. If any data are missing, you’ll need to contact the study’s authors.
2. Your judgment of the quality of the evidence, including risk of bias.

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group.

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

1. Narrative (qualitative): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
2. Quantitative: Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis, which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract: A summary of the review
• Introduction: Including the rationale and objectives
• Methods: Including the selection criteria, search method, data extraction method, and synthesis method
• Results: Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion: Including interpretation of the results and limitations of the review
• Conclusion: The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist.

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews, and/or in a peer-reviewed journal.

Frequently asked questions about systematic reviews

What is a literature review?

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question.

It is often written as part of a thesis, dissertation, or research paper, in order to situate your work in relation to existing knowledge.

What is the difference between a literature review and an annotated bibliography?

A literature review is a survey of credible sources on a topic, often used in dissertations, theses, and research papers. Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other academic texts, with an introduction, a main body, and a conclusion.

An annotated bibliography is a list of source references that has a short description (called an annotation) for each of the sources. It is often assigned as part of the research process for a paper.  

Is a systematic review primary research?

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.