systematic approach to literature review

• Find My Rep

You are here

Systematic Approaches to a Successful Literature Review

• Andrew Booth - The University of Sheffield, UK
• Anthea Sutton - The University of Sheffield, UK
• Mark Clowes - Sheffield University, UK
• Marrissa Martyn-St James - Sheffield University, UK
• Description

The perfect project support for any social sciences student, this edition also includes a new chapter on analysing mixed methods research.

Supplements

Student Resources (Free to access) A literature review starter template to demonstrate the sections you need to include for a successful written review. A source credibility checklist to help you assess and think critically about the sources you choose. A source tracker template to help you keep track of your sources and know what you need to include in your audit trail. A downloadable exercise workbook and suggested answers.  A collated list of tried-and-tested tools , including freely available technologies to help you search databases efficiently, plan your work, and keep track of references. A project diary template and example. A complete glossary of terms. Instructor Resources (Log-in needed) PowerPoint slide templates including 10-15 slides per chapter, which can be downloaded and customized for use in your own presentations.

The Systematic Approaches to a Successful Literature Review (third edition) by Andrew Booth, Anthea Sutton, Mark Clowes and Marrissa Martyn-St James is a comprehensive overview of the entire evidence synthesis process – from selecting the appropriate method for an evidence synthesis topic all the way to the analysis and dissemination of the review. This book is of relevance to anyone interested in evidence synthesis – from trainees to researchers to decision-makers. Anyone can learn something from this book, whether you are a beginner, intermediate, or advance researcher in evidence synthesis. This book is perfect for university-level courses or for anyone interested in evidence synthesis. The exercises, toolbox, key learning points, and frequently asked questions were particularly helpful in advancing my learning.

For our masters level students doing their literature review dissertation this provides effective guidance in approaching their work in a systematic fashion.

Great resource. Easy to read, with helpful tables and diagrams that catch the students' attention and they find easy to recall. The examples and up-to-date links to external sources are also invaluable springboards for the students.

Post-COVID many more students prefer to use the electronic versions of books and the library is also keen to adopt more books in this format, so this is very helpful to enable the maximum number of students to access the helpful text with easy to follow guidance .

I have put this at the top of my reading list for my module on evidence based practice which is like a mini dissertation for level 5 students. It is a comprehensive read and sets out the stages involved in a successful literature review. My students have a problem with this part of the module and this book is a godsend

This book is very important for students to understand how to do an in-depth literature review as a support and motivation for their research.

I did not receive an inspection copy to use

Good comprehensive text - easy to follow

Very clear and useful

Preview this book

For instructors, select a purchasing option.

• Electronic Order Options VitalSource Amazon Kindle Google Play eBooks.com Kobo

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

• 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
• 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
• 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
• PMID: 30089228
• DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

• Guidelines as Topic
• Meta-Analysis as Topic*
• Publication Bias
• Review Literature as Topic
• Systematic Reviews as Topic*

Methodological Approaches to Literature Review

• Living reference work entry
• First Online: 09 May 2023
• Cite this living reference work entry

• Dennis Thomas 2 ,
• Elida Zairina 3 &
• Johnson George 4  

436 Accesses

The literature review can serve various functions in the contexts of education and research. It aids in identifying knowledge gaps, informing research methodology, and developing a theoretical framework during the planning stages of a research study or project, as well as reporting of review findings in the context of the existing literature. This chapter discusses the methodological approaches to conducting a literature review and offers an overview of different types of reviews. There are various types of reviews, including narrative reviews, scoping reviews, and systematic reviews with reporting strategies such as meta-analysis and meta-synthesis. Review authors should consider the scope of the literature review when selecting a type and method. Being focused is essential for a successful review; however, this must be balanced against the relevance of the review to a broad audience.

• Literature review
• Systematic review
• Meta-analysis
• Scoping review
• Research methodology

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Akobeng AK. Principles of evidence based medicine. Arch Dis Child. 2005;90(8):837–40.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Alharbi A, Stevenson M. Refining Boolean queries to identify relevant studies for systematic review updates. J Am Med Inform Assoc. 2020;27(11):1658–66.

Article   PubMed   PubMed Central   Google Scholar  

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.

Article   Google Scholar  

Aromataris E MZE. JBI manual for evidence synthesis. 2020.

Google Scholar  

Aromataris E, Pearson A. The systematic review: an overview. Am J Nurs. 2014;114(3):53–8.

Article   PubMed   Google Scholar  

Aromataris E, Riitano D. Constructing a search strategy and searching for evidence. A guide to the literature search for a systematic review. Am J Nurs. 2014;114(5):49–56.

Babineau J. Product review: covidence (systematic review software). J Canad Health Libr Assoc Canada. 2014;35(2):68–71.

Baker JD. The purpose, process, and methods of writing a literature review. AORN J. 2016;103(3):265–9.

Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.

Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):1–12.

Brown D. A review of the PubMed PICO tool: using evidence-based practice in health education. Health Promot Pract. 2020;21(4):496–8.

Cargo M, Harris J, Pantoja T, et al. Cochrane qualitative and implementation methods group guidance series – paper 4: methods for assessing evidence on intervention implementation. J Clin Epidemiol. 2018;97:59–69.

Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997;126(5):376–80.

Article   CAS   PubMed   Google Scholar  

Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380–7.

Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Cummings SR, Browner WS, Hulley SB, editors. Designing Clinical Research: An Epidemiological Approach. 4th ed. Philadelphia (PA): P Lippincott Williams & Wilkins; 2007. p. 14–22.

Eriksen MB, Frandsen TF. The impact of patient, intervention, comparison, outcome (PICO) as a search strategy tool on literature search quality: a systematic review. JMLA. 2018;106(4):420.

Ferrari R. Writing narrative style literature reviews. Medical Writing. 2015;24(4):230–5.

Flemming K, Booth A, Hannes K, Cargo M, Noyes J. Cochrane qualitative and implementation methods group guidance series – paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses. J Clin Epidemiol. 2018;97:79–85.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26(2):91–108.

Green BN, Johnson CD, Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. J Chiropr Med. 2006;5(3):101–17.

Gregory AT, Denniss AR. An introduction to writing narrative and systematic reviews; tasks, tips and traps for aspiring authors. Heart Lung Circ. 2018;27(7):893–8.

Harden A, Thomas J, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol. 2018;97:70–8.

Harris JL, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 2: methods for question formulation, searching, and protocol development for qualitative evidence synthesis. J Clin Epidemiol. 2018;97:39–48.

Higgins J, Thomas J. In: Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3, updated February 2022). Available from www.training.cochrane.org/handbook.: Cochrane; 2022.

International prospective register of systematic reviews (PROSPERO). Available from https://www.crd.york.ac.uk/prospero/ .

Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118–21.

Landhuis E. Scientific literature: information overload. Nature. 2016;535(7612):457–8.

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. Available from https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03 .

Chapter   Google Scholar  

Lorenzetti DL, Topfer L-A, Dennett L, Clement F. Value of databases other than medline for rapid health technology assessments. Int J Technol Assess Health Care. 2014;30(2):173–8.

Moher D, Liberati A, Tetzlaff J, Altman DG, the PRISMA Group. Preferred reporting items for (SR) and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;6:264–9.

Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ. 1994;309(6954):597–9.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):1–13.

Murphy CM. Writing an effective review article. J Med Toxicol. 2012;8(2):89–90.

NHMRC. Guidelines for guidelines: assessing risk of bias. Available at https://nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias . Last published 29 August 2019. Accessed 29 Aug 2022.

Noyes J, Booth A, Cargo M, et al. Cochrane qualitative and implementation methods group guidance series – paper 1: introduction. J Clin Epidemiol. 2018b;97:35–8.

Noyes J, Booth A, Flemming K, et al. Cochrane qualitative and implementation methods group guidance series – paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings. J Clin Epidemiol. 2018a;97:49–58.

Noyes J, Booth A, Moore G, Flemming K, Tunçalp Ö, Shakibazadeh E. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health. 2019;4(Suppl 1):e000893.

Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Healthcare. 2015;13(3):141–6.

Polanin JR, Pigott TD, Espelage DL, Grotpeter JK. Best practice guidelines for abstract screening large-evidence systematic reviews and meta-analyses. Res Synth Methods. 2019;10(3):330–42.

Article   PubMed Central   Google Scholar  

Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):1–7.

Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. Brit Med J. 2017;358

Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. Br Med J. 2016;355

Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. JAMA. 2000;283(15):2008–12.

Tawfik GM, Dila KAS, Mohamed MYF, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47(1):1–9.

The Critical Appraisal Program. Critical appraisal skills program. Available at https://casp-uk.net/ . 2022. Accessed 29 Aug 2022.

The University of Melbourne. Writing a literature review in Research Techniques 2022. Available at https://students.unimelb.edu.au/academic-skills/explore-our-resources/research-techniques/reviewing-the-literature . Accessed 29 Aug 2022.

The Writing Center University of Winconsin-Madison. Learn how to write a literature review in The Writer’s Handbook – Academic Professional Writing. 2022. Available at https://writing.wisc.edu/handbook/assignments/reviewofliterature/ . Accessed 29 Aug 2022.

Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Stat Med. 1999;18(20):2693–708.

Tricco AC, Lillie E, Zarin W, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Yoneoka D, Henmi M. Clinical heterogeneity in random-effect meta-analysis: between-study boundary estimate problem. Stat Med. 2019;38(21):4131–45.

Yuan Y, Hunt RH. Systematic reviews: the good, the bad, and the ugly. Am J Gastroenterol. 2009;104(5):1086–92.

Download references

Author information

Authors and affiliations.

Centre of Excellence in Treatable Traits, College of Health, Medicine and Wellbeing, University of Newcastle, Hunter Medical Research Institute Asthma and Breathing Programme, Newcastle, NSW, Australia

Dennis Thomas

Department of Pharmacy Practice, Faculty of Pharmacy, Universitas Airlangga, Surabaya, Indonesia

Elida Zairina

Centre for Medicine Use and Safety, Monash Institute of Pharmaceutical Sciences, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Parkville, VIC, Australia

Johnson George

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Johnson George .

Section Editor information

College of Pharmacy, Qatar University, Doha, Qatar

Derek Charles Stewart

Department of Pharmacy, University of Huddersfield, Huddersfield, United Kingdom

Zaheer-Ud-Din Babar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Nature Switzerland AG

About this entry

Cite this entry.

Thomas, D., Zairina, E., George, J. (2023). Methodological Approaches to Literature Review. In: Encyclopedia of Evidence in Pharmaceutical Public Health and Health Services Research in Pharmacy. Springer, Cham. https://doi.org/10.1007/978-3-030-50247-8_57-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-50247-8_57-1

Received : 22 February 2023

Accepted : 22 February 2023

Published : 09 May 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-50247-8

Online ISBN : 978-3-030-50247-8

eBook Packages : Springer Reference Biomedicine and Life Sciences Reference Module Biomedical and Life Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Systematic literature reviews

Systematic approaches to literature review searching.

This guide is primarily for those undertaking a literature review. It outlines how to approach the searching phase systematically in order to identify relevant literature on a research question.

Why be systematic? This approach can:

• Provide a robust overview of the available literature on your topic
• Ensure relevant literature is identified and key publications are not overlooked
• Reduce irrelevant search results through search planning
• Help you to create a reproducible search strategy.

In addition, applying a systematic approach will allow you to work more efficiently.

A literature review may form an essential part of the research process, for example as a major component of a thesis or dissertation. Alternatively, a review may constitute a research project in itself - as a peer-reviewed publication in a journal, or as a report from a research funded project.

While the searching phase of any literature review should be approached in a systematic manner, you do not need to follow all of the techniques outlined in this guide. The methods you choose are dependent on the time and resources you have available, and the purpose of your literature review.

Systematic reviews vs. systematic approaches

A full systematic review aims to comprehensively identify, evaluate and integrate the findings of all relevant studies on a particular research question. A systematic approach involves a rigorous and structured search strategy, without necessarily attempting to include all available research on a particular topic.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Prevent plagiarism. Run a free check.

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved April 2, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Open access
• Published: 14 August 2018

Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies

• Chris Cooper   ORCID: orcid.org/0000-0003-0864-5607 1 ,
• Andrew Booth 2 ,
• Jo Varley-Campbell 1 ,
• Nicky Britten 3 &
• Ruth Garside 4  

BMC Medical Research Methodology volume  18 , Article number:  85 ( 2018 ) Cite this article

199k Accesses

202 Citations

118 Altmetric

Metrics details

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving readers clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before.

The purpose of this review is to determine if a shared model of the literature searching process can be detected across systematic review guidance documents and, if so, how this process is reported in the guidance and supported by published studies.

A literature review.

Two types of literature were reviewed: guidance and published studies. Nine guidance documents were identified, including: The Cochrane and Campbell Handbooks. Published studies were identified through ‘pearl growing’, citation chasing, a search of PubMed using the systematic review methods filter, and the authors’ topic knowledge.

The relevant sections within each guidance document were then read and re-read, with the aim of determining key methodological stages. Methodological stages were identified and defined. This data was reviewed to identify agreements and areas of unique guidance between guidance documents. Consensus across multiple guidance documents was used to inform selection of ‘key stages’ in the process of literature searching.

Eight key stages were determined relating specifically to literature searching in systematic reviews. They were: who should literature search, aims and purpose of literature searching, preparation, the search strategy, searching databases, supplementary searching, managing references and reporting the search process.

Conclusions

Eight key stages to the process of literature searching in systematic reviews were identified. These key stages are consistently reported in the nine guidance documents, suggesting consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews. Further research to determine the suitability of using the same process of literature searching for all types of systematic review is indicated.

Peer Review reports

Systematic literature searching is recognised as a critical component of the systematic review process. It involves a systematic search for studies and aims for a transparent report of study identification, leaving review stakeholders clear about what was done to identify studies, and how the findings of the review are situated in the relevant evidence.

Information specialists and review teams appear to work from a shared and tacit model of the literature search process. How this tacit model has developed and evolved is unclear, and it has not been explicitly examined before. This is in contrast to the information science literature, which has developed information processing models as an explicit basis for dialogue and empirical testing. Without an explicit model, research in the process of systematic literature searching will remain immature and potentially uneven, and the development of shared information models will be assumed but never articulated.

One way of developing such a conceptual model is by formally examining the implicit “programme theory” as embodied in key methodological texts. The aim of this review is therefore to determine if a shared model of the literature searching process in systematic reviews can be detected across guidance documents and, if so, how this process is reported and supported.

Identifying guidance

Key texts (henceforth referred to as “guidance”) were identified based upon their accessibility to, and prominence within, United Kingdom systematic reviewing practice. The United Kingdom occupies a prominent position in the science of health information retrieval, as quantified by such objective measures as the authorship of papers, the number of Cochrane groups based in the UK, membership and leadership of groups such as the Cochrane Information Retrieval Methods Group, the HTA-I Information Specialists’ Group and historic association with such centres as the UK Cochrane Centre, the NHS Centre for Reviews and Dissemination, the Centre for Evidence Based Medicine and the National Institute for Clinical Excellence (NICE). Coupled with the linguistic dominance of English within medical and health science and the science of systematic reviews more generally, this offers a justification for a purposive sample that favours UK, European and Australian guidance documents.

Nine guidance documents were identified. These documents provide guidance for different types of reviews, namely: reviews of interventions, reviews of health technologies, reviews of qualitative research studies, reviews of social science topics, and reviews to inform guidance.

Whilst these guidance documents occasionally offer additional guidance on other types of systematic reviews, we have focused on the core and stated aims of these documents as they relate to literature searching. Table  1 sets out: the guidance document, the version audited, their core stated focus, and a bibliographical pointer to the main guidance relating to literature searching.

Once a list of key guidance documents was determined, it was checked by six senior information professionals based in the UK for relevance to current literature searching in systematic reviews.

Identifying supporting studies

In addition to identifying guidance, the authors sought to populate an evidence base of supporting studies (henceforth referred to as “studies”) that contribute to existing search practice. Studies were first identified by the authors from their knowledge on this topic area and, subsequently, through systematic citation chasing key studies (‘pearls’ [ 1 ]) located within each key stage of the search process. These studies are identified in Additional file  1 : Appendix Table 1. Citation chasing was conducted by analysing the bibliography of references for each study (backwards citation chasing) and through Google Scholar (forward citation chasing). A search of PubMed using the systematic review methods filter was undertaken in August 2017 (see Additional file 1 ). The search terms used were: (literature search*[Title/Abstract]) AND sysrev_methods[sb] and 586 results were returned. These results were sifted for relevance to the key stages in Fig.  1 by CC.

The key stages of literature search guidance as identified from nine key texts

Extracting the data

To reveal the implicit process of literature searching within each guidance document, the relevant sections (chapters) on literature searching were read and re-read, with the aim of determining key methodological stages. We defined a key methodological stage as a distinct step in the overall process for which specific guidance is reported, and action is taken, that collectively would result in a completed literature search.

The chapter or section sub-heading for each methodological stage was extracted into a table using the exact language as reported in each guidance document. The lead author (CC) then read and re-read these data, and the paragraphs of the document to which the headings referred, summarising section details. This table was then reviewed, using comparison and contrast to identify agreements and areas of unique guidance. Consensus across multiple guidelines was used to inform selection of ‘key stages’ in the process of literature searching.

Having determined the key stages to literature searching, we then read and re-read the sections relating to literature searching again, extracting specific detail relating to the methodological process of literature searching within each key stage. Again, the guidance was then read and re-read, first on a document-by-document-basis and, secondly, across all the documents above, to identify both commonalities and areas of unique guidance.

Results and discussion

Our findings.

We were able to identify consensus across the guidance on literature searching for systematic reviews suggesting a shared implicit model within the information retrieval community. Whilst the structure of the guidance varies between documents, the same key stages are reported, even where the core focus of each document is different. We were able to identify specific areas of unique guidance, where a document reported guidance not summarised in other documents, together with areas of consensus across guidance.

Unique guidance

Only one document provided guidance on the topic of when to stop searching [ 2 ]. This guidance from 2005 anticipates a topic of increasing importance with the current interest in time-limited (i.e. “rapid”) reviews. Quality assurance (or peer review) of literature searches was only covered in two guidance documents [ 3 , 4 ]. This topic has emerged as increasingly important as indicated by the development of the PRESS instrument [ 5 ]. Text mining was discussed in four guidance documents [ 4 , 6 , 7 , 8 ] where the automation of some manual review work may offer efficiencies in literature searching [ 8 ].

Agreement between guidance: Defining the key stages of literature searching

Where there was agreement on the process, we determined that this constituted a key stage in the process of literature searching to inform systematic reviews.

From the guidance, we determined eight key stages that relate specifically to literature searching in systematic reviews. These are summarised at Fig. 1 . The data extraction table to inform Fig. 1 is reported in Table  2 . Table 2 reports the areas of common agreement and it demonstrates that the language used to describe key stages and processes varies significantly between guidance documents.

For each key stage, we set out the specific guidance, followed by discussion on how this guidance is situated within the wider literature.

Key stage one: Deciding who should undertake the literature search

The guidance.

Eight documents provided guidance on who should undertake literature searching in systematic reviews [ 2 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ]. The guidance affirms that people with relevant expertise of literature searching should ‘ideally’ be included within the review team [ 6 ]. Information specialists (or information scientists), librarians or trial search co-ordinators (TSCs) are indicated as appropriate researchers in six guidance documents [ 2 , 7 , 8 , 9 , 10 , 11 ].

How the guidance corresponds to the published studies

The guidance is consistent with studies that call for the involvement of information specialists and librarians in systematic reviews [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 ] and which demonstrate how their training as ‘expert searchers’ and ‘analysers and organisers of data’ can be put to good use [ 13 ] in a variety of roles [ 12 , 16 , 20 , 21 , 24 , 25 , 26 ]. These arguments make sense in the context of the aims and purposes of literature searching in systematic reviews, explored below. The need for ‘thorough’ and ‘replicable’ literature searches was fundamental to the guidance and recurs in key stage two. Studies have found poor reporting, and a lack of replicable literature searches, to be a weakness in systematic reviews [ 17 , 18 , 27 , 28 ] and they argue that involvement of information specialists/ librarians would be associated with better reporting and better quality literature searching. Indeed, Meert et al. [ 29 ] demonstrated that involving a librarian as a co-author to a systematic review correlated with a higher score in the literature searching component of a systematic review [ 29 ]. As ‘new styles’ of rapid and scoping reviews emerge, where decisions on how to search are more iterative and creative, a clear role is made here too [ 30 ].

Knowing where to search for studies was noted as important in the guidance, with no agreement as to the appropriate number of databases to be searched [ 2 , 6 ]. Database (and resource selection more broadly) is acknowledged as a relevant key skill of information specialists and librarians [ 12 , 15 , 16 , 31 ].

Whilst arguments for including information specialists and librarians in the process of systematic review might be considered self-evident, Koffel and Rethlefsen [ 31 ] have questioned if the necessary involvement is actually happening [ 31 ].

Key stage two: Determining the aim and purpose of a literature search

The aim: Five of the nine guidance documents use adjectives such as ‘thorough’, ‘comprehensive’, ‘transparent’ and ‘reproducible’ to define the aim of literature searching [ 6 , 7 , 8 , 9 , 10 ]. Analogous phrases were present in a further three guidance documents, namely: ‘to identify the best available evidence’ [ 4 ] or ‘the aim of the literature search is not to retrieve everything. It is to retrieve everything of relevance’ [ 2 ] or ‘A systematic literature search aims to identify all publications relevant to the particular research question’ [ 3 ]. The Joanna Briggs Institute reviewers’ manual was the only guidance document where a clear statement on the aim of literature searching could not be identified. The purpose of literature searching was defined in three guidance documents, namely to minimise bias in the resultant review [ 6 , 8 , 10 ]. Accordingly, eight of nine documents clearly asserted that thorough and comprehensive literature searches are required as a potential mechanism for minimising bias.

The need for thorough and comprehensive literature searches appears as uniform within the eight guidance documents that describe approaches to literature searching in systematic reviews of effectiveness. Reviews of effectiveness (of intervention or cost), accuracy and prognosis, require thorough and comprehensive literature searches to transparently produce a reliable estimate of intervention effect. The belief that all relevant studies have been ‘comprehensively’ identified, and that this process has been ‘transparently’ reported, increases confidence in the estimate of effect and the conclusions that can be drawn [ 32 ]. The supporting literature exploring the need for comprehensive literature searches focuses almost exclusively on reviews of intervention effectiveness and meta-analysis. Different ‘styles’ of review may have different standards however; the alternative, offered by purposive sampling, has been suggested in the specific context of qualitative evidence syntheses [ 33 ].

What is a comprehensive literature search?

Whilst the guidance calls for thorough and comprehensive literature searches, it lacks clarity on what constitutes a thorough and comprehensive literature search, beyond the implication that all of the literature search methods in Table 2 should be used to identify studies. Egger et al. [ 34 ], in an empirical study evaluating the importance of comprehensive literature searches for trials in systematic reviews, defined a comprehensive search for trials as:

a search not restricted to English language;

where Cochrane CENTRAL or at least two other electronic databases had been searched (such as MEDLINE or EMBASE); and

at least one of the following search methods has been used to identify unpublished trials: searches for (I) conference abstracts, (ii) theses, (iii) trials registers; and (iv) contacts with experts in the field [ 34 ].

Tricco et al. (2008) used a similar threshold of bibliographic database searching AND a supplementary search method in a review when examining the risk of bias in systematic reviews. Their criteria were: one database (limited using the Cochrane Highly Sensitive Search Strategy (HSSS)) and handsearching [ 35 ].

Together with the guidance, this would suggest that comprehensive literature searching requires the use of BOTH bibliographic database searching AND supplementary search methods.

Comprehensiveness in literature searching, in the sense of how much searching should be undertaken, remains unclear. Egger et al. recommend that ‘investigators should consider the type of literature search and degree of comprehension that is appropriate for the review in question, taking into account budget and time constraints’ [ 34 ]. This view tallies with the Cochrane Handbook, which stipulates clearly, that study identification should be undertaken ‘within resource limits’ [ 9 ]. This would suggest that the limitations to comprehension are recognised but it raises questions on how this is decided and reported [ 36 ].

What is the point of comprehensive literature searching?

The purpose of thorough and comprehensive literature searches is to avoid missing key studies and to minimize bias [ 6 , 8 , 10 , 34 , 37 , 38 , 39 ] since a systematic review based only on published (or easily accessible) studies may have an exaggerated effect size [ 35 ]. Felson (1992) sets out potential biases that could affect the estimate of effect in a meta-analysis [ 40 ] and Tricco et al. summarize the evidence concerning bias and confounding in systematic reviews [ 35 ]. Egger et al. point to non-publication of studies, publication bias, language bias and MEDLINE bias, as key biases [ 34 , 35 , 40 , 41 , 42 , 43 , 44 , 45 , 46 ]. Comprehensive searches are not the sole factor to mitigate these biases but their contribution is thought to be significant [ 2 , 32 , 34 ]. Fehrmann (2011) suggests that ‘the search process being described in detail’ and that, where standard comprehensive search techniques have been applied, increases confidence in the search results [ 32 ].

Does comprehensive literature searching work?

Egger et al., and other study authors, have demonstrated a change in the estimate of intervention effectiveness where relevant studies were excluded from meta-analysis [ 34 , 47 ]. This would suggest that missing studies in literature searching alters the reliability of effectiveness estimates. This is an argument for comprehensive literature searching. Conversely, Egger et al. found that ‘comprehensive’ searches still missed studies and that comprehensive searches could, in fact, introduce bias into a review rather than preventing it, through the identification of low quality studies then being included in the meta-analysis [ 34 ]. Studies query if identifying and including low quality or grey literature studies changes the estimate of effect [ 43 , 48 ] and question if time is better invested updating systematic reviews rather than searching for unpublished studies [ 49 ], or mapping studies for review as opposed to aiming for high sensitivity in literature searching [ 50 ].

Aim and purpose beyond reviews of effectiveness

The need for comprehensive literature searches is less certain in reviews of qualitative studies, and for reviews where a comprehensive identification of studies is difficult to achieve (for example, in Public health) [ 33 , 51 , 52 , 53 , 54 , 55 ]. Literature searching for qualitative studies, and in public health topics, typically generates a greater number of studies to sift than in reviews of effectiveness [ 39 ] and demonstrating the ‘value’ of studies identified or missed is harder [ 56 ], since the study data do not typically support meta-analysis. Nussbaumer-Streit et al. (2016) have registered a review protocol to assess whether abbreviated literature searches (as opposed to comprehensive literature searches) has an impact on conclusions across multiple bodies of evidence, not only on effect estimates [ 57 ] which may develop this understanding. It may be that decision makers and users of systematic reviews are willing to trade the certainty from a comprehensive literature search and systematic review in exchange for different approaches to evidence synthesis [ 58 ], and that comprehensive literature searches are not necessarily a marker of literature search quality, as previously thought [ 36 ]. Different approaches to literature searching [ 37 , 38 , 59 , 60 , 61 , 62 ] and developing the concept of when to stop searching are important areas for further study [ 36 , 59 ].

The study by Nussbaumer-Streit et al. has been published since the submission of this literature review [ 63 ]. Nussbaumer-Streit et al. (2018) conclude that abbreviated literature searches are viable options for rapid evidence syntheses, if decision-makers are willing to trade the certainty from a comprehensive literature search and systematic review, but that decision-making which demands detailed scrutiny should still be based on comprehensive literature searches [ 63 ].

Key stage three: Preparing for the literature search

Six documents provided guidance on preparing for a literature search [ 2 , 3 , 6 , 7 , 9 , 10 ]. The Cochrane Handbook clearly stated that Cochrane authors (i.e. researchers) should seek advice from a trial search co-ordinator (i.e. a person with specific skills in literature searching) ‘before’ starting a literature search [ 9 ].

Two key tasks were perceptible in preparing for a literature searching [ 2 , 6 , 7 , 10 , 11 ]. First, to determine if there are any existing or on-going reviews, or if a new review is justified [ 6 , 11 ]; and, secondly, to develop an initial literature search strategy to estimate the volume of relevant literature (and quality of a small sample of relevant studies [ 10 ]) and indicate the resources required for literature searching and the review of the studies that follows [ 7 , 10 ].

Three documents summarised guidance on where to search to determine if a new review was justified [ 2 , 6 , 11 ]. These focused on searching databases of systematic reviews (The Cochrane Database of Systematic Reviews (CDSR) and the Database of Abstracts of Reviews of Effects (DARE)), institutional registries (including PROSPERO), and MEDLINE [ 6 , 11 ]. It is worth noting, however, that as of 2015, DARE (and NHS EEDs) are no longer being updated and so the relevance of this (these) resource(s) will diminish over-time [ 64 ]. One guidance document, ‘Systematic reviews in the Social Sciences’, noted, however, that databases are not the only source of information and unpublished reports, conference proceeding and grey literature may also be required, depending on the nature of the review question [ 2 ].

Two documents reported clearly that this preparation (or ‘scoping’) exercise should be undertaken before the actual search strategy is developed [ 7 , 10 ]).

The guidance offers the best available source on preparing the literature search with the published studies not typically reporting how their scoping informed the development of their search strategies nor how their search approaches were developed. Text mining has been proposed as a technique to develop search strategies in the scoping stages of a review although this work is still exploratory [ 65 ]. ‘Clustering documents’ and word frequency analysis have also been tested to identify search terms and studies for review [ 66 , 67 ]. Preparing for literature searches and scoping constitutes an area for future research.

Key stage four: Designing the search strategy

The Population, Intervention, Comparator, Outcome (PICO) structure was the commonly reported structure promoted to design a literature search strategy. Five documents suggested that the eligibility criteria or review question will determine which concepts of PICO will be populated to develop the search strategy [ 1 , 4 , 7 , 8 , 9 ]. The NICE handbook promoted multiple structures, namely PICO, SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) and multi-stranded approaches [ 4 ].

With the exclusion of The Joanna Briggs Institute reviewers’ manual, the guidance offered detail on selecting key search terms, synonyms, Boolean language, selecting database indexing terms and combining search terms. The CEE handbook suggested that ‘search terms may be compiled with the help of the commissioning organisation and stakeholders’ [ 10 ].

The use of limits, such as language or date limits, were discussed in all documents [ 2 , 3 , 4 , 6 , 7 , 8 , 9 , 10 , 11 ].

Search strategy structure

The guidance typically relates to reviews of intervention effectiveness so PICO – with its focus on intervention and comparator - is the dominant model used to structure literature search strategies [ 68 ]. PICOs – where the S denotes study design - is also commonly used in effectiveness reviews [ 6 , 68 ]. As the NICE handbook notes, alternative models to structure literature search strategies have been developed and tested. Booth provides an overview on formulating questions for evidence based practice [ 69 ] and has developed a number of alternatives to the PICO structure, namely: BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) for use when systematically identifying theory [ 55 ]; SPICE (Setting, Perspective, Intervention, Comparison, Evaluation) for identification of social science and evaluation studies [ 69 ] and, working with Cooke and colleagues, SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) [ 70 ]. SPIDER has been compared to PICO and PICOs in a study by Methley et al. [ 68 ].

The NICE handbook also suggests the use of multi-stranded approaches to developing literature search strategies [ 4 ]. Glanville developed this idea in a study by Whitting et al. [ 71 ] and a worked example of this approach is included in the development of a search filter by Cooper et al. [ 72 ].

Writing search strategies: Conceptual and objective approaches

Hausner et al. [ 73 ] provide guidance on writing literature search strategies, delineating between conceptually and objectively derived approaches. The conceptual approach, advocated by and explained in the guidance documents, relies on the expertise of the literature searcher to identify key search terms and then develop key terms to include synonyms and controlled syntax. Hausner and colleagues set out the objective approach [ 73 ] and describe what may be done to validate it [ 74 ].

The use of limits

The guidance documents offer direction on the use of limits within a literature search. Limits can be used to focus literature searching to specific study designs or by other markers (such as by date) which limits the number of studies returned by a literature search. The use of limits should be described and the implications explored [ 34 ] since limiting literature searching can introduce bias (explored above). Craven et al. have suggested the use of a supporting narrative to explain decisions made in the process of developing literature searches and this advice would usefully capture decisions on the use of search limits [ 75 ].

Key stage five: Determining the process of literature searching and deciding where to search (bibliographic database searching)

Table 2 summarises the process of literature searching as reported in each guidance document. Searching bibliographic databases was consistently reported as the ‘first step’ to literature searching in all nine guidance documents.

Three documents reported specific guidance on where to search, in each case specific to the type of review their guidance informed, and as a minimum requirement [ 4 , 9 , 11 ]. Seven of the key guidance documents suggest that the selection of bibliographic databases depends on the topic of review [ 2 , 3 , 4 , 6 , 7 , 8 , 10 ], with two documents noting the absence of an agreed standard on what constitutes an acceptable number of databases searched [ 2 , 6 ].

The guidance documents summarise ‘how to’ search bibliographic databases in detail and this guidance is further contextualised above in terms of developing the search strategy. The documents provide guidance of selecting bibliographic databases, in some cases stating acceptable minima (i.e. The Cochrane Handbook states Cochrane CENTRAL, MEDLINE and EMBASE), and in other cases simply listing bibliographic database available to search. Studies have explored the value in searching specific bibliographic databases, with Wright et al. (2015) noting the contribution of CINAHL in identifying qualitative studies [ 76 ], Beckles et al. (2013) questioning the contribution of CINAHL to identifying clinical studies for guideline development [ 77 ], and Cooper et al. (2015) exploring the role of UK-focused bibliographic databases to identify UK-relevant studies [ 78 ]. The host of the database (e.g. OVID or ProQuest) has been shown to alter the search returns offered. Younger and Boddy [ 79 ] report differing search returns from the same database (AMED) but where the ‘host’ was different [ 79 ].

The average number of bibliographic database searched in systematic reviews has risen in the period 1994–2014 (from 1 to 4) [ 80 ] but there remains (as attested to by the guidance) no consensus on what constitutes an acceptable number of databases searched [ 48 ]. This is perhaps because thinking about the number of databases searched is the wrong question, researchers should be focused on which databases were searched and why, and which databases were not searched and why. The discussion should re-orientate to the differential value of sources but researchers need to think about how to report this in studies to allow findings to be generalised. Bethel (2017) has proposed ‘search summaries’, completed by the literature searcher, to record where included studies were identified, whether from database (and which databases specifically) or supplementary search methods [ 81 ]. Search summaries document both yield and accuracy of searches, which could prospectively inform resource use and decisions to search or not to search specific databases in topic areas. The prospective use of such data presupposes, however, that past searches are a potential predictor of future search performance (i.e. that each topic is to be considered representative and not unique). In offering a body of practice, this data would be of greater practicable use than current studies which are considered as little more than individual case studies [ 82 , 83 , 84 , 85 , 86 , 87 , 88 , 89 , 90 ].

When to database search is another question posed in the literature. Beyer et al. [ 91 ] report that databases can be prioritised for literature searching which, whilst not addressing the question of which databases to search, may at least bring clarity as to which databases to search first [ 91 ]. Paradoxically, this links to studies that suggest PubMed should be searched in addition to MEDLINE (OVID interface) since this improves the currency of systematic reviews [ 92 , 93 ]. Cooper et al. (2017) have tested the idea of database searching not as a primary search method (as suggested in the guidance) but as a supplementary search method in order to manage the volume of studies identified for an environmental effectiveness systematic review. Their case study compared the effectiveness of database searching versus a protocol using supplementary search methods and found that the latter identified more relevant studies for review than searching bibliographic databases [ 94 ].

Key stage six: Determining the process of literature searching and deciding where to search (supplementary search methods)

Table 2 also summaries the process of literature searching which follows bibliographic database searching. As Table 2 sets out, guidance that supplementary literature search methods should be used in systematic reviews recurs across documents, but the order in which these methods are used, and the extent to which they are used, varies. We noted inconsistency in the labelling of supplementary search methods between guidance documents.

Rather than focus on the guidance on how to use the methods (which has been summarised in a recent review [ 95 ]), we focus on the aim or purpose of supplementary search methods.

The Cochrane Handbook reported that ‘efforts’ to identify unpublished studies should be made [ 9 ]. Four guidance documents [ 2 , 3 , 6 , 9 ] acknowledged that searching beyond bibliographic databases was necessary since ‘databases are not the only source of literature’ [ 2 ]. Only one document reported any guidance on determining when to use supplementary methods. The IQWiG handbook reported that the use of handsearching (in their example) could be determined on a ‘case-by-case basis’ which implies that the use of these methods is optional rather than mandatory. This is in contrast to the guidance (above) on bibliographic database searching.

The issue for supplementary search methods is similar in many ways to the issue of searching bibliographic databases: demonstrating value. The purpose and contribution of supplementary search methods in systematic reviews is increasingly acknowledged [ 37 , 61 , 62 , 96 , 97 , 98 , 99 , 100 , 101 ] but understanding the value of the search methods to identify studies and data is unclear. In a recently published review, Cooper et al. (2017) reviewed the literature on supplementary search methods looking to determine the advantages, disadvantages and resource implications of using supplementary search methods [ 95 ]. This review also summarises the key guidance and empirical studies and seeks to address the question on when to use these search methods and when not to [ 95 ]. The guidance is limited in this regard and, as Table 2 demonstrates, offers conflicting advice on the order of searching, and the extent to which these search methods should be used in systematic reviews.

Key stage seven: Managing the references

Five of the documents provided guidance on managing references, for example downloading, de-duplicating and managing the output of literature searches [ 2 , 4 , 6 , 8 , 10 ]. This guidance typically itemised available bibliographic management tools rather than offering guidance on how to use them specifically [ 2 , 4 , 6 , 8 ]. The CEE handbook provided guidance on importing data where no direct export option is available (e.g. web-searching) [ 10 ].

The literature on using bibliographic management tools is not large relative to the number of ‘how to’ videos on platforms such as YouTube (see for example [ 102 ]). These YouTube videos confirm the overall lack of ‘how to’ guidance identified in this study and offer useful instruction on managing references. Bramer et al. set out methods for de-duplicating data and reviewing references in Endnote [ 103 , 104 ] and Gall tests the direct search function within Endnote to access databases such as PubMed, finding a number of limitations [ 105 ]. Coar et al. and Ahmed et al. consider the role of the free-source tool, Zotero [ 106 , 107 ]. Managing references is a key administrative function in the process of review particularly for documenting searches in PRISMA guidance.

Key stage eight: Documenting the search

The Cochrane Handbook was the only guidance document to recommend a specific reporting guideline: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [ 9 ]. Six documents provided guidance on reporting the process of literature searching with specific criteria to report [ 3 , 4 , 6 , 8 , 9 , 10 ]. There was consensus on reporting: the databases searched (and the host searched by), the search strategies used, and any use of limits (e.g. date, language, search filters (The CRD handbook called for these limits to be justified [ 6 ])). Three guidance documents reported that the number of studies identified should be recorded [ 3 , 6 , 10 ]. The number of duplicates identified [ 10 ], the screening decisions [ 3 ], a comprehensive list of grey literature sources searched (and full detail for other supplementary search methods) [ 8 ], and an annotation of search terms tested but not used [ 4 ] were identified as unique items in four documents.

The Cochrane Handbook was the only guidance document to note that the full search strategies for each database should be included in the Additional file 1 of the review [ 9 ].

All guidance documents should ultimately deliver completed systematic reviews that fulfil the requirements of the PRISMA reporting guidelines [ 108 ]. The guidance broadly requires the reporting of data that corresponds with the requirements of the PRISMA statement although documents typically ask for diverse and additional items [ 108 ]. In 2008, Sampson et al. observed a lack of consensus on reporting search methods in systematic reviews [ 109 ] and this remains the case as of 2017, as evidenced in the guidance documents, and in spite of the publication of the PRISMA guidelines in 2009 [ 110 ]. It is unclear why the collective guidance does not more explicitly endorse adherence to the PRISMA guidance.

Reporting of literature searching is a key area in systematic reviews since it sets out clearly what was done and how the conclusions of the review can be believed [ 52 , 109 ]. Despite strong endorsement in the guidance documents, specifically supported in PRISMA guidance, and other related reporting standards too (such as ENTREQ for qualitative evidence synthesis, STROBE for reviews of observational studies), authors still highlight the prevalence of poor standards of literature search reporting [ 31 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 ]. To explore issues experienced by authors in reporting literature searches, and look at uptake of PRISMA, Radar et al. [ 120 ] surveyed over 260 review authors to determine common problems and their work summaries the practical aspects of reporting literature searching [ 120 ]. Atkinson et al. [ 121 ] have also analysed reporting standards for literature searching, summarising recommendations and gaps for reporting search strategies [ 121 ].

One area that is less well covered by the guidance, but nevertheless appears in this literature, is the quality appraisal or peer review of literature search strategies. The PRESS checklist is the most prominent and it aims to develop evidence-based guidelines to peer review of electronic search strategies [ 5 , 122 , 123 ]. A corresponding guideline for documentation of supplementary search methods does not yet exist although this idea is currently being explored.

How the reporting of the literature searching process corresponds to critical appraisal tools is an area for further research. In the survey undertaken by Radar et al. (2014), 86% of survey respondents (153/178) identified a need for further guidance on what aspects of the literature search process to report [ 120 ]. The PRISMA statement offers a brief summary of what to report but little practical guidance on how to report it [ 108 ]. Critical appraisal tools for systematic reviews, such as AMSTAR 2 (Shea et al. [ 124 ]) and ROBIS (Whiting et al. [ 125 ]), can usefully be read alongside PRISMA guidance, since they offer greater detail on how the reporting of the literature search will be appraised and, therefore, they offer a proxy on what to report [ 124 , 125 ]. Further research in the form of a study which undertakes a comparison between PRISMA and quality appraisal checklists for systematic reviews would seem to begin addressing the call, identified by Radar et al., for further guidance on what to report [ 120 ].

Limitations

Other handbooks exist.

A potential limitation of this literature review is the focus on guidance produced in Europe (the UK specifically) and Australia. We justify the decision for our selection of the nine guidance documents reviewed in this literature review in section “ Identifying guidance ”. In brief, these nine guidance documents were selected as the most relevant health care guidance that inform UK systematic reviewing practice, given that the UK occupies a prominent position in the science of health information retrieval. We acknowledge the existence of other guidance documents, such as those from North America (e.g. the Agency for Healthcare Research and Quality (AHRQ) [ 126 ], The Institute of Medicine [ 127 ] and the guidance and resources produced by the Canadian Agency for Drugs and Technologies in Health (CADTH) [ 128 ]). We comment further on this directly below.

The handbooks are potentially linked to one another

What is not clear is the extent to which the guidance documents inter-relate or provide guidance uniquely. The Cochrane Handbook, first published in 1994, is notably a key source of reference in guidance and systematic reviews beyond Cochrane reviews. It is not clear to what extent broadening the sample of guidance handbooks to include North American handbooks, and guidance handbooks from other relevant countries too, would alter the findings of this literature review or develop further support for the process model. Since we cannot be clear, we raise this as a potential limitation of this literature review. On our initial review of a sample of North American, and other, guidance documents (before selecting the guidance documents considered in this review), however, we do not consider that the inclusion of these further handbooks would alter significantly the findings of this literature review.

This is a literature review

A further limitation of this review was that the review of published studies is not a systematic review of the evidence for each key stage. It is possible that other relevant studies could help contribute to the exploration and development of the key stages identified in this review.

This literature review would appear to demonstrate the existence of a shared model of the literature searching process in systematic reviews. We call this model ‘the conventional approach’, since it appears to be common convention in nine different guidance documents.

The findings reported above reveal eight key stages in the process of literature searching for systematic reviews. These key stages are consistently reported in the nine guidance documents which suggests consensus on the key stages of literature searching, and therefore the process of literature searching as a whole, in systematic reviews.

In Table 2 , we demonstrate consensus regarding the application of literature search methods. All guidance documents distinguish between primary and supplementary search methods. Bibliographic database searching is consistently the first method of literature searching referenced in each guidance document. Whilst the guidance uniformly supports the use of supplementary search methods, there is little evidence for a consistent process with diverse guidance across documents. This may reflect differences in the core focus across each document, linked to differences in identifying effectiveness studies or qualitative studies, for instance.

Eight of the nine guidance documents reported on the aims of literature searching. The shared understanding was that literature searching should be thorough and comprehensive in its aim and that this process should be reported transparently so that that it could be reproduced. Whilst only three documents explicitly link this understanding to minimising bias, it is clear that comprehensive literature searching is implicitly linked to ‘not missing relevant studies’ which is approximately the same point.

Defining the key stages in this review helps categorise the scholarship available, and it prioritises areas for development or further study. The supporting studies on preparing for literature searching (key stage three, ‘preparation’) were, for example, comparatively few, and yet this key stage represents a decisive moment in literature searching for systematic reviews. It is where search strategy structure is determined, search terms are chosen or discarded, and the resources to be searched are selected. Information specialists, librarians and researchers, are well placed to develop these and other areas within the key stages we identify.

This review calls for further research to determine the suitability of using the conventional approach. The publication dates of the guidance documents which underpin the conventional approach may raise questions as to whether the process which they each report remains valid for current systematic literature searching. In addition, it may be useful to test whether it is desirable to use the same process model of literature searching for qualitative evidence synthesis as that for reviews of intervention effectiveness, which this literature review demonstrates is presently recommended best practice.

Abbreviations

Behaviour of interest; Health context; Exclusions; Models or Theories

Cochrane Database of Systematic Reviews

The Cochrane Central Register of Controlled Trials

Database of Abstracts of Reviews of Effects

Enhancing transparency in reporting the synthesis of qualitative research

Institute for Quality and Efficiency in Healthcare

National Institute for Clinical Excellence

Population, Intervention, Comparator, Outcome

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Setting, Perspective, Intervention, Comparison, Evaluation

Sample, Phenomenon of Interest, Design, Evaluation, Research type

STrengthening the Reporting of OBservational studies in Epidemiology

Trial Search Co-ordinators

Booth A. Unpacking your literature search toolbox: on search styles and tactics. Health Information & Libraries Journal. 2008;25(4):313–7.

Article   Google Scholar  

Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Oxford: Blackwell Publishing Ltd; 2006.

Book   Google Scholar  

Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG Methods Resources. 7 Information retrieval 2014 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK385787/ .

NICE: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual 2014. Available from: https://www.nice.org.uk/media/default/about/what-we-do/our-programmes/developing-nice-guidelines-the-manual.pdf .

Sampson M. MJ, Lefebvre C, Moher D, Grimshaw J. Peer Review of Electronic Search Strategies: PRESS; 2008.

Google Scholar  

Centre for Reviews & Dissemination. Systematic reviews – CRD’s guidance for undertaking reviews in healthcare. York: Centre for Reviews and Dissemination, University of York; 2009.

eunetha: European Network for Health Technology Assesment Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness 2016. Available from: http://www.eunethta.eu/sites/default/files/Guideline_Information_Retrieval_V1-1.pdf .

Kugley SWA, Thomas J, Mahood Q, Jørgensen AMK, Hammerstrøm K, Sathe N. Searching for studies: a guide to information retrieval for Campbell systematic reviews. Oslo: Campbell Collaboration. 2017; Available from: https://www.campbellcollaboration.org/library/searching-for-studies-information-retrieval-guide-campbell-reviews.html

Lefebvre C, Manheimer E, Glanville J. Chapter 6: searching for studies. In: JPT H, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions; 2011.

Collaboration for Environmental Evidence. Guidelines for Systematic Review and Evidence Synthesis in Environmental Management.: Environmental Evidence:; 2013. Available from: http://www.environmentalevidence.org/wp-content/uploads/2017/01/Review-guidelines-version-4.2-final-update.pdf .

The Joanna Briggs Institute. Joanna Briggs institute reviewers’ manual. 2014th ed: the Joanna Briggs institute; 2014. Available from: https://joannabriggs.org/assets/docs/sumari/ReviewersManual-2014.pdf

Beverley CA, Booth A, Bath PA. The role of the information specialist in the systematic review process: a health information case study. Health Inf Libr J. 2003;20(2):65–74.

Article   CAS   Google Scholar  

Harris MR. The librarian's roles in the systematic review process: a case study. Journal of the Medical Library Association. 2005;93(1):81–7.

PubMed   PubMed Central   Google Scholar  

Egger JB. Use of recommended search strategies in systematic reviews and the impact of librarian involvement: a cross-sectional survey of recent authors. PLoS One. 2015;10(5):e0125931.

Li L, Tian J, Tian H, Moher D, Liang F, Jiang T, et al. Network meta-analyses could be improved by searching more sources and by involving a librarian. J Clin Epidemiol. 2014;67(9):1001–7.

Article   PubMed   Google Scholar  

McGowan J, Sampson M. Systematic reviews need systematic searchers. J Med Libr Assoc. 2005;93(1):74–80.

Rethlefsen ML, Farrell AM, Osterhaus Trzasko LC, Brigham TJ. Librarian co-authors correlated with higher quality reported search strategies in general internal medicine systematic reviews. J Clin Epidemiol. 2015;68(6):617–26.

Weller AC. Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports. J Med Libr Assoc. 2004;92(2):163–4.

Swinkels A, Briddon J, Hall J. Two physiotherapists, one librarian and a systematic literature review: collaboration in action. Health Info Libr J. 2006;23(4):248–56.

Foster M. An overview of the role of librarians in systematic reviews: from expert search to project manager. EAHIL. 2015;11(3):3–7.

Lawson L. OPERATING OUTSIDE LIBRARY WALLS 2004.

Vassar M, Yerokhin V, Sinnett PM, Weiher M, Muckelrath H, Carr B, et al. Database selection in systematic reviews: an insight through clinical neurology. Health Inf Libr J. 2017;34(2):156–64.

Townsend WA, Anderson PF, Ginier EC, MacEachern MP, Saylor KM, Shipman BL, et al. A competency framework for librarians involved in systematic reviews. Journal of the Medical Library Association : JMLA. 2017;105(3):268–75.

Cooper ID, Crum JA. New activities and changing roles of health sciences librarians: a systematic review, 1990-2012. Journal of the Medical Library Association : JMLA. 2013;101(4):268–77.

Crum JA, Cooper ID. Emerging roles for biomedical librarians: a survey of current practice, challenges, and changes. Journal of the Medical Library Association : JMLA. 2013;101(4):278–86.

Dudden RF, Protzko SL. The systematic review team: contributions of the health sciences librarian. Med Ref Serv Q. 2011;30(3):301–15.

Golder S, Loke Y, McIntosh HM. Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008;61(5):440–8.

Maggio LA, Tannery NH, Kanter SL. Reproducibility of literature search reporting in medical education reviews. Academic medicine : journal of the Association of American Medical Colleges. 2011;86(8):1049–54.

Meert D, Torabi N, Costella J. Impact of librarians on reporting of the literature searching component of pediatric systematic reviews. Journal of the Medical Library Association : JMLA. 2016;104(4):267–77.

Morris M, Boruff JT, Gore GC. Scoping reviews: establishing the role of the librarian. Journal of the Medical Library Association : JMLA. 2016;104(4):346–54.

Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016;11(9):e0163309.

Article   PubMed   PubMed Central   CAS   Google Scholar  

Fehrmann P, Thomas J. Comprehensive computer searches and reporting in systematic reviews. Research Synthesis Methods. 2011;2(1):15–32.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Systematic Reviews. 2016;5(1):74.

Article   PubMed   PubMed Central   Google Scholar  

Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health technology assessment (Winchester, England). 2003;7(1):1–76.

Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61(5):422–34.

Booth A. How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010;26(4):431–5.

Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Inf Libr J. 2010;27(2):114–22.

Petticrew M. Time to rethink the systematic review catechism? Moving from ‘what works’ to ‘what happens’. Systematic Reviews. 2015;4(1):36.

Betrán AP, Say L, Gülmezoglu AM, Allen T, Hampson L. Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality. BMC Med Res Methodol. 2005;5

Felson DT. Bias in meta-analytic research. J Clin Epidemiol. 1992;45(8):885–92.

Article   PubMed   CAS   Google Scholar  

Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345(6203):1502–5.

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews. BMC Med Res Methodol. 2017;17(1):64.

Schmucker CM, Blümle A, Schell LK, Schwarzer G, Oeller P, Cabrera L, et al. Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS One. 2017;12(4):e0176210.

Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet (London, England). 1997;350(9074):326–9.

Moher D, Pham B, Lawson ML, Klassen TP. The inclusion of reports of randomised trials published in languages other than English in systematic reviews. Health technology assessment (Winchester, England). 2003;7(41):1–90.

Pham B, Klassen TP, Lawson ML, Moher D. Language of publication restrictions in systematic reviews gave different results depending on whether the intervention was conventional or complementary. J Clin Epidemiol. 2005;58(8):769–76.

Mills EJ, Kanters S, Thorlund K, Chaimani A, Veroniki A-A, Ioannidis JPA. The effects of excluding treatments from network meta-analyses: survey. BMJ : British Medical Journal. 2013;347

Hartling L, Featherstone R, Nuspl M, Shave K, Dryden DM, Vandermeer B. The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Med Res Methodol. 2016;16(1):127.

van Driel ML, De Sutter A, De Maeseneer J, Christiaens T. Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009;62(8):838–44.e3.

Buchberger B, Krabbe L, Lux B, Mattivi JT. Evidence mapping for decision making: feasibility versus accuracy - when to abandon high sensitivity in electronic searches. German medical science : GMS e-journal. 2016;14:Doc09.

Lorenc T, Pearson M, Jamal F, Cooper C, Garside R. The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Research Synthesis Methods. 2012;3(1):1–10.

Gough D. Weight of evidence: a framework for the appraisal of the quality and relevance of evidence. Res Pap Educ. 2007;22(2):213–28.

Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153–78.

Britten N, Garside R, Pope C, Frost J, Cooper C. Asking more of qualitative synthesis: a response to Sally Thorne. Qual Health Res. 2017;27(9):1370–6.

Booth A, Carroll C. Systematic searching for theory to inform systematic reviews: is it feasible? Is it desirable? Health Info Libr J. 2015;32(3):220–35.

Kwon Y, Powelson SE, Wong H, Ghali WA, Conly JM. An assessment of the efficacy of searching in biomedical databases beyond MEDLINE in identifying studies for a systematic review on ward closures as an infection control intervention to control outbreaks. Syst Rev. 2014;3:135.

Nussbaumer-Streit B, Klerings I, Wagner G, Titscher V, Gartlehner G. Assessing the validity of abbreviated literature searches for rapid reviews: protocol of a non-inferiority and meta-epidemiologic study. Systematic Reviews. 2016;5:197.

Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017;17(1):121.

Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804–8.

Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(4):591–603.

Pearson M, Moxham T, Ashton K. Effectiveness of search strategies for qualitative research about barriers and facilitators of program delivery. Eval Health Prof. 2011;34(3):297–308.

Levay P, Raynor M, Tuvey D. The Contributions of MEDLINE, Other Bibliographic Databases and Various Search Techniques to NICE Public Health Guidance. 2015. 2015;10(1):19.

Nussbaumer-Streit B, Klerings I, Wagner G, Heise TL, Dobrescu AI, Armijo-Olivo S, et al. Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study. J Clin Epidemiol. 2018;102:1–11.

Briscoe S, Cooper C, Glanville J, Lefebvre C. The loss of the NHS EED and DARE databases and the effect on evidence synthesis and evaluation. Res Synth Methods. 2017;8(3):256–7.

Stansfield C, O'Mara-Eves A, Thomas J. Text mining for search term development in systematic reviewing: A discussion of some methods and challenges. Research Synthesis Methods.n/a-n/a.

Petrova M, Sutcliffe P, Fulford KW, Dale J. Search terms and a validated brief search filter to retrieve publications on health-related values in Medline: a word frequency analysis study. Journal of the American Medical Informatics Association : JAMIA. 2012;19(3):479–88.

Stansfield C, Thomas J, Kavanagh J. 'Clustering' documents automatically to support scoping reviews of research: a case study. Res Synth Methods. 2013;4(3):230–41.

PubMed   Google Scholar  

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

Andrew B. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355–68.

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22(10):1435–43.

Whiting P, Westwood M, Bojke L, Palmer S, Richardson G, Cooper J, et al. Clinical effectiveness and cost-effectiveness of tests for the diagnosis and investigation of urinary tract infection in children: a systematic review and economic model. Health technology assessment (Winchester, England). 2006;10(36):iii-iv, xi-xiii, 1–154.

Cooper C, Levay P, Lorenc T, Craig GM. A population search filter for hard-to-reach populations increased search efficiency for a systematic review. J Clin Epidemiol. 2014;67(5):554–9.

Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Systematic Reviews. 2012;1(1):19.

Hausner E, Guddat C, Hermanns T, Lampert U, Waffenschmidt S. Prospective comparison of search strategies for systematic reviews: an objective approach yielded higher sensitivity than a conceptual one. J Clin Epidemiol. 2016;77:118–24.

Craven J, Levay P. Recording database searches for systematic reviews - what is the value of adding a narrative to peer-review checklists? A case study of nice interventional procedures guidance. Evid Based Libr Inf Pract. 2011;6(4):72–87.

Wright K, Golder S, Lewis-Light K. What value is the CINAHL database when searching for systematic reviews of qualitative studies? Syst Rev. 2015;4:104.

Beckles Z, Glover S, Ashe J, Stockton S, Boynton J, Lai R, et al. Searching CINAHL did not add value to clinical questions posed in NICE guidelines. J Clin Epidemiol. 2013;66(9):1051–7.

Cooper C, Rogers M, Bethel A, Briscoe S, Lowe J. A mapping review of the literature on UK-focused health and social care databases. Health Inf Libr J. 2015;32(1):5–22.

Younger P, Boddy K. When is a search not a search? A comparison of searching the AMED complementary health database via EBSCOhost, OVID and DIALOG. Health Inf Libr J. 2009;26(2):126–35.

Lam MT, McDiarmid M. Increasing number of databases searched in systematic reviews and meta-analyses between 1994 and 2014. Journal of the Medical Library Association : JMLA. 2016;104(4):284–9.

Bethel A, editor Search summary tables for systematic reviews: results and findings. HLC Conference 2017a.

Aagaard T, Lund H, Juhl C. Optimizing literature search in systematic reviews - are MEDLINE, EMBASE and CENTRAL enough for identifying effect studies within the area of musculoskeletal disorders? BMC Med Res Methodol. 2016;16(1):161.

Adams CE, Frederick K. An investigation of the adequacy of MEDLINE searches for randomized controlled trials (RCTs) of the effects of mental health care. Psychol Med. 1994;24(3):741–8.

Kelly L, St Pierre-Hansen N. So many databases, such little clarity: searching the literature for the topic aboriginal. Canadian family physician Medecin de famille canadien. 2008;54(11):1572–3.

Lawrence DW. What is lost when searching only one literature database for articles relevant to injury prevention and safety promotion? Injury Prevention. 2008;14(6):401–4.

Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta-analysis of observational studies. J Clin Epidemiol. 2005;58(9):867–73.

Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, et al. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55.

Stevinson C, Lawlor DA. Searching multiple databases for systematic reviews: added value or diminishing returns? Complementary Therapies in Medicine. 2004;12(4):228–32.

Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476–87.

Taylor B, Wylie E, Dempster M, Donnelly M. Systematically retrieving research: a case study evaluating seven databases. Res Soc Work Pract. 2007;17(6):697–706.

Beyer FR, Wright K. Can we prioritise which databases to search? A case study using a systematic review of frozen shoulder management. Health Info Libr J. 2013;30(1):49–58.

Duffy S, de Kock S, Misso K, Noake C, Ross J, Stirk L. Supplementary searches of PubMed to improve currency of MEDLINE and MEDLINE in-process searches via Ovid. Journal of the Medical Library Association : JMLA. 2016;104(4):309–12.

Katchamart W, Faulkner A, Feldman B, Tomlinson G, Bombardier C. PubMed had a higher sensitivity than Ovid-MEDLINE in the search for systematic reviews. J Clin Epidemiol. 2011;64(7):805–7.

Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplementary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement (in Press). Research Synthesis Methods. 2017;

Cooper C, Booth, A., Britten, N., Garside, R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: A methodological review. (In Press). BMC Systematic Reviews. 2017.

Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ (Clinical research ed). 2005;331(7524):1064–5.

Article   PubMed Central   Google Scholar  

Hinde S, Spackman E. Bidirectional citation searching to completion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5–11.

Levay P, Ainsworth N, Kettle R, Morgan A. Identifying evidence for public health guidance: a comparison of citation searching with web of science and Google scholar. Res Synth Methods. 2016;7(1):34–45.

McManus RJ, Wilson S, Delaney BC, Fitzmaurice DA, Hyde CJ, Tobias RS, et al. Review of the usefulness of contacting other experts when conducting a literature search for systematic reviews. BMJ (Clinical research ed). 1998;317(7172):1562–3.

Westphal A, Kriston L, Holzel LP, Harter M, von Wolff A. Efficiency and contribution of strategies for finding randomized controlled trials: a case study from a systematic review on therapeutic interventions of chronic depression. Journal of public health research. 2014;3(2):177.

Matthews EJ, Edwards AG, Barker J, Bloor M, Covey J, Hood K, et al. Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care. Health Libr Rev. 1999;16(2):112–20.

Bethel A. Endnote Training (YouTube Videos) 2017b [Available from: http://medicine.exeter.ac.uk/esmi/workstreams/informationscience/is_resources,_guidance_&_advice/ .

Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240–3.

Bramer WM, Milic J, Mast F. Reviewing retrieved references for inclusion in systematic reviews using EndNote. Journal of the Medical Library Association : JMLA. 2017;105(1):84–7.

Gall C, Brahmi FA. Retrieval comparison of EndNote to search MEDLINE (Ovid and PubMed) versus searching them directly. Medical reference services quarterly. 2004;23(3):25–32.

Ahmed KK, Al Dhubaib BE. Zotero: a bibliographic assistant to researcher. J Pharmacol Pharmacother. 2011;2(4):303–5.

Coar JT, Sewell JP. Zotero: harnessing the power of a personal bibliographic manager. Nurse Educ. 2010;35(5):205–7.

Moher D, Liberati A, Tetzlaff J, Altman DG, The PG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.

Sampson M, McGowan J, Tetzlaff J, Cogo E, Moher D. No consensus exists on search reporting methods for systematic reviews. J Clin Epidemiol. 2008;61(8):748–54.

Toews LC. Compliance of systematic reviews in veterinary journals with preferred reporting items for systematic reviews and meta-analysis (PRISMA) literature search reporting guidelines. Journal of the Medical Library Association : JMLA. 2017;105(3):233–9.

Booth A. "brimful of STARLITE": toward standards for reporting literature searches. Journal of the Medical Library Association : JMLA. 2006;94(4):421–9. e205

Faggion CM Jr, Wu YC, Tu YK, Wasiak J. Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery. Br J Radiol. 2016;89(1062):20150878.

Mullins MM, DeLuca JB, Crepaz N, Lyles CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000–2010): are the searches clearly explained, systematic and reproducible? Research Synthesis Methods. 2014;5(2):116–30.

Yoshii A, Plaut DA, McGraw KA, Anderson MJ, Wellik KE. Analysis of the reporting of search strategies in Cochrane systematic reviews. Journal of the Medical Library Association : JMLA. 2009;97(1):21–9.

Bigna JJ, Um LN, Nansseu JR. A comparison of quality of abstracts of systematic reviews including meta-analysis of randomized controlled trials in high-impact general medicine journals before and after the publication of PRISMA extension for abstracts: a systematic review and meta-analysis. Syst Rev. 2016;5(1):174.

Akhigbe T, Zolnourian A, Bulters D. Compliance of systematic reviews articles in brain arteriovenous malformation with PRISMA statement guidelines: review of literature. Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia. 2017;39:45–8.

Tao KM, Li XQ, Zhou QH, Moher D, Ling CQ, Yu WF. From QUOROM to PRISMA: a survey of high-impact medical journals' instructions to authors and a review of systematic reviews in anesthesia literature. PLoS One. 2011;6(11):e27611.

Wasiak J, Tyack Z, Ware R. Goodwin N. Jr. Poor methodological quality and reporting standards of systematic reviews in burn care management. International wound journal: Faggion CM; 2016.

Tam WW, Lo KK, Khalechelvam P. Endorsement of PRISMA statement and quality of systematic reviews and meta-analyses published in nursing journals: a cross-sectional study. BMJ Open. 2017;7(2):e013905.

Rader T, Mann M, Stansfield C, Cooper C, Sampson M. Methods for documenting systematic review searches: a discussion of common issues. Res Synth Methods. 2014;5(2):98–115.

Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for literature searches and report inclusion criteria: making research syntheses more transparent and easy to replicate. Res Synth Methods. 2015;6(1):87–95.

McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62(9):944–52.

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ (Clinical research ed). 2017;358.

Whiting P, Savović J, Higgins JPT, Caldwell DM, Reeves BC, Shea B, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol. 2016;69:225–34.

Relevo R, Balshem H. Finding evidence for comparing medical interventions: AHRQ and the effective health care program. J Clin Epidemiol. 2011;64(11):1168–77.

Medicine Io. Standards for Systematic Reviews 2011 [Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx .

CADTH: Resources 2018.

Download references

Acknowledgements

CC acknowledges the supervision offered by Professor Chris Hyde.

This publication forms a part of CC’s PhD. CC’s PhD was funded through the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) Programme (Project Number 16/54/11). The open access fee for this publication was paid for by Exeter Medical School.

RG and NB were partially supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South West Peninsula.

The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Author information

Authors and affiliations.

Institute of Health Research, University of Exeter Medical School, Exeter, UK

Chris Cooper & Jo Varley-Campbell

HEDS, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Andrew Booth

Nicky Britten

European Centre for Environment and Human Health, University of Exeter Medical School, Truro, UK

Ruth Garside

You can also search for this author in PubMed   Google Scholar

Contributions

CC conceived the idea for this study and wrote the first draft of the manuscript. CC discussed this publication in PhD supervision with AB and separately with JVC. CC revised the publication with input and comments from AB, JVC, RG and NB. All authors revised the manuscript prior to submission. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chris Cooper .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:.

Appendix tables and PubMed search strategy. Key studies used for pearl growing per key stage, working data extraction tables and the PubMed search strategy. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Cooper, C., Booth, A., Varley-Campbell, J. et al. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies. BMC Med Res Methodol 18 , 85 (2018). https://doi.org/10.1186/s12874-018-0545-3

Download citation

Received : 20 September 2017

Accepted : 06 August 2018

Published : 14 August 2018

DOI : https://doi.org/10.1186/s12874-018-0545-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Literature Search Process
• Citation Chasing
• Tacit Models
• Unique Guidance
• Information Specialists

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

• Open access
• Published: 20 September 2023

Older adults’ experiences during the COVID-19 pandemic: a qualitative systematic literature review

• Elfriede Derrer-Merk   ORCID: orcid.org/0000-0001-7241-0808 1 ,
• Maria-Fernanda Reyes-Rodriguez   ORCID: orcid.org/0000-0002-2645-5092 2 ,
• Laura K. Soulsby   ORCID: orcid.org/0000-0001-9071-8654 1 ,
• Louise Roper   ORCID: orcid.org/0000-0002-2918-7628 3 &
• Kate M. Bennett   ORCID: orcid.org/0000-0003-3164-6894 1  

BMC Geriatrics volume  23 , Article number:  580 ( 2023 ) Cite this article

1537 Accesses

5 Citations

2 Altmetric

Metrics details

Relatively little is known about the lived experiences of older adults during the COVID-19 pandemic. We systematically review the international literature to understand the lived experiences of older adult’s experiences during the pandemic.

Design and methodology

This study uses a meta-ethnographical approach to investigate the included studies. The analyses were undertaken with constructivist grounded theory.

Thirty-two studies met the inclusion criteria and only five papers were of low quality. Most, but not all studies, were from the global north. We identified three themes: desired and challenged wellbeing; coping and adaptation; and discrimination and intersectionality.

Overall, the studies’ findings were varied and reflected different times during the pandemic. Studies reported the impact of mass media messaging and its mostly negative impact on older adults. Many studies highlighted the impact of the COVID-19 pandemic on participants' social connectivity and well-being including missing the proximity of loved ones and in consequence experienced an increase in anxiety, feeling of depression, or loneliness. However, many studies reported how participants adapted to the change of lifestyle including new ways of communication, and social distancing. Some studies focused on discrimination and the experiences of sexual and gender minority and ethnic minority participants. Studies found that the pandemic impacted the participants’ well-being including suicidal risk behaviour, friendship loss, and increased mental health issues.

The COVID-19 pandemic disrupted and impacted older adults’ well-being worldwide. Despite the cultural and socio-economic differences many commonalities were found. Studies described the impact of mass media reporting, social connectivity, impact of confinement on well-being, coping, and on discrimination. The authors suggest that these findings need to be acknowledged for future pandemic strategies. Additionally, policy-making processes need to include older adults to address their needs. PROSPERO record [CRD42022331714], (Derrer-Merk et al., Older adults’ lived experiences during the COVID-19 pandemic: a systematic review, 2022).

Peer Review reports

Introduction

In March 2020 the World Health Organisation declared a pandemic caused by the virus SARS-CoV2 (COVID-19) [ 1 ]. At this time 118,000 cases in 114 countries were identified and 4,291 people had already lost their lives [ 2 ]. By July 2022, there were over 5.7 million active cases and over 6.4 million deaths [ 2 ]. Despite the effort to combat and eliminate the virus globally, new variants of the virus are still a concern. At the start of the pandemic, little was known about who would be most at risk, but emerging data suggested that both people with underlying health conditions and older people had a higher risk of becoming seriously ill [ 3 ]. Thus, countries worldwide imposed health and safety measures aimed at reducing viral transmission and protecting people at higher risk of contracting the virus [ 4 ]. These measures included: national lockdowns with different lengths and frequencies; targeted shopping times for older people; hygiene procedures (wearing masks, washing hands regularly, disinfecting hands); restricting or prohibiting social gatherings; working from home, school closure, and home-schooling.

Research suggests that lockdowns and protective measures impacted on people’s lives, and had a particular impact on older people. They were at higher risk from COVID-19, with greater disease severity and higher mortality compared to younger people [ 5 ]. Older adults were identified as at higher risk as they are more likely to have pre-existing conditions including heart disease, diabetes, and severe respiratory conditions [ 5 ]. Additionally, recent research highlights that COVID-19 and its safety measures led to increased mental health problems, including increased feelings of depression, anxiety, social isolation, and loneliness, potentially cognitive decline [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ]. Other studies reported the consequences of only age-based protective health measures including self-isolation for people older people (e.g. feeling old, losing out the time with family) [ 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 ].

Over the past decade, the World Health Organisation (WHO) has recognised the importance of risk communication within public health emergency preparedness and response, especially in the context of epidemics and pandemics. Risk communication is defined as “the real-time exchange of information, advice and opinions between experts or officials, and people who face a threat (hazard) to their survival, health or economic or social well-being” ([ 31 ], p5). This includes reporting the risk and health protection measurements through media and governmental bodies. Constructing awareness and building trust in society are essential components of risk communication [ 32 ]. In the context of the pandemic, the WHO noted that individual risk perception helped to prompt problem-solving activities (such as wearing face masks, social distancing, and self-isolation). However, the prolonged perception of pandemic-related uncertainty and risk could also lead to heightened feelings of distress and anxiety [ 31 , 33 ], see also [ 34 , 35 , 36 , 37 ].

This new and unprecedented disease provided the ground for researchers worldwide to investigate the COVID-19 pandemic. To date (August 2022), approximately 8072 studies have been recorded on the U.S. National Library of Medicine ClinicalTrials.gov [ 38 ] and 12002 systematic reviews have been registered at PROSPERO, concerning COVID-19. However, to our knowledge, there is little known about qualitative research as a response to the COVID-19 pandemic and how it impacted older adults’ well-being [ 39 ]. In particular, little is known about how older people experienced the pandemic. Thus, our research question considers: How did older adults experience the COVID-19 pandemic worldwide?

We use a qualitative evidence synthesis (QES) recommended by Cochrane Qualitative and Implementation Methods Group to identify peer-reviewed articles [ 40 ]. This provides an overview of existing research, identifies potential research gaps, and develops new cumulative knowledge concerning the COVID-19 pandemic and older adults’ experiences. QES is a valuable method for its potential to contribute to research and policy [ 41 ]. Flemming and Noyes [ 40 ] argue that the evidence synthesis from qualitative research provides a richer interpretation compared to single primary research. They identified an increasing demand for qualitative evidence synthesis from a wide range of “health and social professionals, policymakers, guideline developers and educationalists” (p.1).

Methodology

A systematic literature review requires a specific approach compared to other reviews. Although there is no consensus on how it is conducted, recent systematic literature reviews have agreed the following reporting criteria are addressed [ 42 , 43 ]: (a) a research question; (b) reporting database, and search strategy; (c) inclusion and exclusion criteria; (d) reporting selection methods; (e) critically appraisal tools; (f) data analysis and synthesis. We applied these criteria in our study and began by registering the research protocol with Prospero [ 44 ].

The study is registered at Prospero [ 44 ]. This systematic literature review incorporates qualitative studies concerning older adults’ experiences during the COVID-19 pandemic.

Search strategy

The primary qualitative articles were identified via a systematic search as per the qualitative-specific SPIDER approach [ 45 ]. The SPIDER tool is designed to structure qualitative research questions, focusing less on interventions and more on study design, and ‘samples’ rather than populations, encompassing:

S-Sample. This includes all articles concerning older adults aged 60 +  [ 1 ].

P-Phenomena of Interest. How did older adults experience the COVID-19 pandemic?

D-Design. We aim to investigate qualitative studies concerning the experiences of older adults during the COVID-19 pandemic.

E-Evaluation. The evaluation of studies will be evaluated with the amended Critical Appraisal Skills Programme CASP [ 46 ].

R-Research type Qualitative

Information source

The following databases were searched: PsychInfo, Medline, CINAHL, Web of Science, Annual Review, Annual Review of Gerontology, and Geriatrics. A hand search was conducted on Google Scholar and additional searches examined the reference lists of the included papers. The keyword search included the following terms: (older adults or elderly) AND (COVID-19 or SARS or pandemic) AND (experiences); (older adults) AND (experience) AND (covid-19) OR (coronavirus); (older adults) AND (experience) AND (covid-19 OR coronavirus) AND (Qualitative). Additional hand search terms included e.g. senior, senior citizen, or old age.

Inclusion and exclusion criteria

Articles were included when they met the following criteria: primary research using qualitative methods related to the lived experience of older adults aged 60 + (i.e. the experiences of individuals during the COVID-19 pandemic); peer-reviewed journal articles published in English; related to the COVID-19 pandemic; empirical research; published from 2020 till August 2022.

Articles were excluded when: papers discussed health professionals’ experiences; diagnostics; medical studies; interventions; day-care; home care; or carers; experiences with dementia; studies including hospitals; quantitative studies; mixed-method studies; single-case studies; people under the age of 60; grey literature; scoping reviews, and systematic reviews. We excluded clinical/care-related studies as we wanted to explore the everyday experiences of people aged 60 + . Mixed-method studies were excluded as we were interested in what was represented in solely qualitative studies. However, we acknowledge, that mixed-method studies are valuable for future systematic reviews.

Meta-ethnography

The qualitative synthesis was undertaken by using meta-ethnography. The authors have chosen meta-ethnography over other methodologies as it is an inductive and interpretive synthesis analysis and is uniquely “suited to developing new conceptual models and theories” ([ 47 ], p 2), see also [ 48 ]. Therefore, it combines well with constructivist grounded theory methodology. Meta-ethnography also examines and identifies areas of disagreements between studies [ 48 ].

This is of particular interest as the lived experiences of older adults during the COVID-19 pandemic were likely to be diverse. The method enables the researcher to synthesise the findings (e.g. themes, concepts) from primary studies, acknowledging primary data (quotes) by “using a unique translation synthesis method to transcend the findings of individual study accounts and create higher order” constructs ([ 47 ], p. 2). The following seven steps were applied:

Getting started (identify area of interest). We were interested in the lived experiences of older adults worldwide.

Deciding what was relevant to the initial interest (defining the focus, locating relevant studies, decision to include studies, quality appraisal). We decided on the inclusion and exclusion criteria and an appropriate quality appraisal.

Reading the studies. We used the screening process described below (title, abstract, full text)

Determining how the studies were related (extracting first-order constructs- participants’ quotes and second-order construct- primary author interpretation, clustering the themes from the studies into new categories (Table 3 ).

Translating the studies into one another (comparing and contrasting the studies, checking commonalities or differences of each article) to organise and develop higher-order constructs by using constant comparison (Table 3 ). Translating is the process of finding commonalities between studies [ 48 ].

Synthesising the translation (reciprocal and refutational synthesis, a lines of argument synthesis (interpretation of the relationship between the themes- leads to key themes and constructs of higher order; creating new meaning, Tables 2 , 3 ),

Expressing the synthesis (writing up the findings) [ 47 , 48 ].

Screening and Study Selection

A 4-stage screening protocol was followed (Fig.  1 Prisma). First, all selected studies were screened for duplicates, which were deleted. Second, all remaining studies were screened for eligibility, and non-relevant studies were excluded at the preliminary stage. These screening steps were as follows: 1. title screening; 2. abstract screening, by the first and senior authors independently; and 3. full-text screening which was undertaken for almost all papers by the first author. However, 2 papers [ 9 , 23 ] were assessed independently by LS, LR, and LMM to avoid a conflict of interest. The other co-authors also screened independently a portion of the papers each, to ensure that each paper had two independent screens to determine inclusion in the review [ 49 ]. This avoided bias and confirmed the eligibility of the included papers (Fig.  1 ). Endnote reference management was used to store the articles and aid the screening process.

Prisma flow diagram adapted from Page et al. [ 50 ]. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 )

Data extraction

After title and abstract screening, 39 papers were selected for reading the full article. 7 papers were excluded after the full-text assessment (1 study was conducted in 2017, but published in 2021; 2 papers were not fully available in English, 2 papers did not address the research question, 1 article was based on a conference abstract only, 1 article had only one participant age 65 +).

The full-text screening included 32 studies. All the included studies, alongside the CASP template, data extraction table, the draft of this article, and translation for synthesising the findings [ 47 , 48 ] were available and accessible on google drive for all co-authors. All authors discussed the findings in regular meetings.

Quality appraisal

A critical appraisal tool assesses a study for its trustworthiness, methodological rigor, and biases and ensures “transparency in the assessment of primary research” ([ 51 ], p. 5); see also [ 48 , 49 , 50 , 51 , 52 , 53 ]. There is currently no gold standard for assessing primary qualitative studies, but different authors agreed that the amended CASPS checklist was appropriate to assess qualitative studies [ 46 , 54 ]. Thus, we use the amended CASP appraisal tool [ 42 ]. The amended CASP appraisal tool aims to improve qualitative evidence synthesis by assessing ontology and epistemology (Table 1 CASP appraisal tool).

A numerical score was assigned to each question to indicate whether the criteria had been met (= 2), partially met (= 1), or not met (= 0) [ 54 ]; see also [ 55 ]. The score 16 – 22 are considered to be moderate and high-quality studies. The studies scored 15 and below were identified as low-quality papers. Although we focus on higher-quality papers, we did not exclude papers to avoid the exclusion of insightful and meaningful data [ 42 , 48 , 52 , 53 , 54 , 55 , 56 , 57 ]. The quality of the paper was considered in developing the evidence synthesis.

We followed the appraisal questions applied for each included study and answered the criteria either ‘Yes’, ‘Cannot tell’, or ‘No’. (Table 1 CASP appraisal criteria). The tenth question asking the value of the article was answered with ‘high’ of importance, ‘middle’, or low of importance. The new eleventh question in the CASP tool concerning ontology and epistemology was answered with yes, no, or partly (Table 1 ).

Data synthesis

The data synthesis followed the seven steps of Meta-Ethnography developed by Noblit & Hare [ 58 ], starting the data synthesis at step 3, described in detail by [ 47 ]. This encompasses: reading the studies; determining how the studies are related; translating the studies into one another; synthesis the translations; and expressing synthesis. This review provides a synthesis of the findings from studies related to the experiences of older adults during the COVID-19 pandemic. The qualitative analyses are based on constructivist grounded theory [ 59 ] to identify the experiences of older adults during the COVID-19 pandemic (non-clinical) populations. The analysis is inductive and iterative, uses constant comparison, and aims to develop a theory. The qualitative synthesis encompasses all text labelled as ‘results’ or ‘findings’ and uses this as raw data. The raw data includes participant’s quotes; thus, the synthesis is grounded in the participant's experience [ 47 , 48 , 60 , 61 ]. The initial coding was undertaken for each eligible article line by line. Please see Table 2 Themes per author and country. Focused coding was applied using constant comparison, which is a widely used approach in grounded theory [ 61 ]. In particular, common and recurring as well as contradicting concepts within the studies were identified, clustered into categories, and overarching higher order constructs were developed [ 47 , 48 , 60 ] (Tables 2 , 3 , 4 ).

We identified twenty-seven out of thirty-two studies as moderate-high quality; they met most of the criteria (scoring 16/22 or above on the CASP; [ 54 ]. Only five papers were identified as low qualitative papers scoring 15 and below [ 71 , 73 , 74 , 86 , 91 ]. Please see the scores provided for each paper in Table 4 . The low-quality papers did not provide sufficient details regarding the researcher’s relationship with the participants, sampling and recruitment, data collection, rigor in the analysis, or epistemological or ontological reasoning. For example, Yildirim [ 91 ] used verbatim notes as data without recording or transcribing them. This article described the analytical process briefly but was missing a discussion of the applied reflexivity of using verbatim notes and its limitations [ 92 ].

This systematic review found that many studies did not mention the relationship between the authors and the participant. The CASP critical appraisal tool asks: Has the relationship between the researcher and participants been adequately considered? (reflecting on own role, potential bias). Many studies reported that the recruitment was drawn from larger studies and that the qualitative study was a sub-study. Others reported that participants contacted the researcher after advertising the study. One study Goins et al., [ 72 ] reported that students recruited family members, but did not discuss how this potential bias impacted the results.

Our review brings new insights into older adults’ experiences during the pandemic worldwide. The studies were conducted on almost all continents. The majority of the articles were written in Europe followed by North America and Canada (4: USA; 3: Canada, UK; 2: Brazil, India, Netherlands, Sweden, Turkey 2; 1: Austria, China, Finland, India/Iran, Mauritius, New Zealand, Serbia, Spain, Switzerland, Uganda, UK/Ireland, UK/Colombia) (see Fig.  2 ). Note, as the review focuses on English language publications, we are unable to comment on qualitative research conducted in other languages see [ 72 ].

Numbers of publications by country

The characteristics of the included studies and the presence of analytical themes can be found in Table 4 . We used the following characteristics: Author and year of publication, research aims, the country conducted, Participant’s age, number of participants, analytical methodology, CASP score, and themes.

We identified three themes: desired and challenged wellbeing; coping and adaptation; discrimination and intersectionality. We will discuss the themes in turn.

Desired and challenged wellbeing

Most of the studies reported the impact of the COVID-19 pandemic on the well-being of older adults. Factors which influenced wellbeing included: risk communication and risk perception; social connectivity; confinement (at home); and means of coping and adapting. In this context, well-being refers to the evidence reported about participants' physical and mental health, and social connectivity.

Risk perception and risk communication

Politicians and media transmitted messages about the response to the pandemic to the public worldwide. These included mortality and morbidity reports, and details of health and safety regulations like social distancing, shielding- self-isolation, or wearing masks [ 34 , 35 , 36 , 37 ]. As this risk communication is crucial to combat the spread of the virus, it is also important to understand how people perceived the reporting during the pandemic.

Seven studies reported on how the mass media impacted participants' well-being [ 23 , 67 , 68 , 70 , 72 , 81 , 85 ]. Sangrar et al. [ 68 ] investigated how older adults responded to COVID-19 messaging: “My reaction was to try to make sure that I listen to everything and [I] made sure I was aware of all the suggestions and the precautions that were being expressed by various agencies …”. (p. 4). Other studies reported the negative impact on participants' well-being of constant messaging and as a consequence stopped watching the news to maintain emotional well-being [ 3 , 67 , 68 , 70 , 72 , 81 , 85 ]. Derrer-Merk et al. [ 23 ] reported one participant said that “At first, watching the news every day is depressing and getting more and more depressing by the day, so I’ve had to stop watching it for my own peace of mind” (p. 13). In addition, news reporting impacted participants’ risk perception. For example, “Sometimes we are scared to hear the huge coverage of COVID-19 news, in particular the repeated message ‘older is risky’, although the message is useful.” ([ 81 ], p5).

• Social connectivity

Social connectivity and support from family and community were found in fourteen of the studies as important themes [ 9 , 62 , 66 , 67 , 68 , 75 , 76 , 77 , 78 , 79 , 80 , 83 , 84 , 90 ].

The impact of COVID-19 on social networks highlighted the diverse experiences of participants. Some participants reported that the size of social contact was reduced: “We have been quite isolated during this corona time” ?([ 80 ], p. 3). Whilst other participants reported that the network was stable except that the method of contact was different: “These friends and relatives, they visited and called as often as before, but of course, we needed to use the telephone when it was not possible to meet” ([ 77 ], p. 5). Many participants in this study did not want to expand their social network see also [ 9 , 77 , 78 , 79 ]. Hafford-Letchfield et al. [ 76 ] reported that established social networks and relationships were beneficial for the participants: “Covid has affected our relationship (with partner), we spend some really positive close time together and support each other a lot” (p. 7).

On the other hand, other studies reported decreases of, and gaps in, social connectedness: “I couldn’t do a lot of things that I’ve been doing for years. That was playing competitive badminton three times a week, I couldn’t do that. I couldn’t get up early and go volunteer in Seattle” [ 9 , 67 , 75 ]. A loss of social connection with children and grandchildren was often mentioned: “We cannot see our grandchildren up close and personal because, well because they [the parents] don’t want us, they don’t want to risk our being with the kids … it’s been an emotional loss exacerbated by the COVID thing” ([ 68 ] p.10); see also [ 9 , 67 , 78 ]. On the contrary, Chemen & Gopalla [ 66 ] note that those older adults who were living with other family members reported that they were more valued: “Last night my daughter-in-law thanked me for helping with my granddaughter” (p.4).

Despite reports of social disconnectedness, some studies highlighted the importance of support from family members and how support changed during the COVID-19 pandemic [ 9 , 62 , 81 , 83 , 90 ]. Yang et al. [ 90 ] argued that social support was essential during the Lockdown in China: “N6 said: ‘I asked my son-in-law to take me to the hospital” (p. 4810). Mahapatra et al. [ 81 ] found, in an Indian study, that the complex interplay of support on different levels (individual, family, and community) helped participants to adapt to the new situation. For example, this participant reported that: “The local police are very helpful. When I rang them for something and asked them to find out about it, they responded immediately” (p. 5).

Impact of confinement on well being

Most articles highlighted the impact of confinement on older adults’ well-being [ 9 , 62 , 63 , 65 , 67 , 69 , 70 , 72 , 75 , 77 , 78 , 79 , 81 , 82 , 83 , 85 , 89 , 90 ].

Some studies found that participants maintained emotional well-being during the pandemic and it did not change their lifestyle [ 79 , 80 , 82 , 83 , 89 , 92 ]: “Actually, I used this crisis period to clean my house. Bookcases are completely cleaned and I discarded old books. Well, we have actually been very busy with those kind of jobs. So, we were not bored at all” ([ 79 ], p. 5). In McKinlay et al. [ 82 ]’s study, nearly half of the participants found that having a sense of purpose helped to maintain their well-being: “You have to have a purpose you see. I think mental resilience is all about having a sense of purpose” (p. 6).

However, at the same time, the majority of the articles (12 out of 18) highlighted the negative impact of confinement and social distancing. Participants talked of increased depressive feelings and anxiety. For example, one of Akkus et al.’s [ 62 ] participants said: “... I am depressed; people died. Terrible disease does not give up, it always kills, I am afraid of it …” (p. 549). Similarly, one of Falvo et al.’s [ 67 ] participants remarked: “I am locked inside my house and I am afraid to go out” (p. 7).

Many of the studies reported the negative impact of loneliness as a result of confinement on participants’ well-being including [ 69 , 70 , 72 , 78 , 79 , 90 , 93 ]. Falvo et al. [ 67 ] reported that many participants experienced loneliness: “What sense does it make when you are not even able to see a family member? I mean, it is the saddest thing not to have the comfort of having your family next to you, to be really alone” (p. 8).

Not all studies found a negative impact on loneliness. For example, a “loner advantage” was found by Xie et al. ([ 82 ], p. 386). In this study participants found benefits in already being alone “It’s just a part of who I am, and I think that helps—if you can be alone, it really is an asset when you have to be alone” ([ 82 ], p. 386).

Bundy et al. [ 80 ] investigated loneliness from already lonely older adults and found that many participants did not attribute the loneliness to the pandemic: “It’s not been a whole lot, because I was already sitting around the house a whole lot anyway ( …). It’s basically the same, pretty well … I’d pretty well be like this anyway with COVID or without COVID” (p. 873) (see also [ 83 ]).

A study from Serbia investigated how the curfew was perceived 15 months afterward. Some participants were calm: “I realized that … well … it was simply necessary. For that reason, we accepted it as a measure that is for the common good” ([ 70 ], p.634). Others were shocked: “Above all, it was a huge surprise and sort of a shock, a complete shock because I have never, ever seen it in my life and I felt horrible, because I thought that something even worse is coming, that I even could not fathom” ([ 70 ], p. 634).

The lockdowns brought not only mental health issues to the fore but impacted the physical health of participants. Some reported they were fearful of the COVID-19 pandemic: “... For a little while I was afraid to leave, to go outside. I didn’t know if you got it from the air” ([ 75 ]. p. 6). Another study reported: “It’s been important for me to walk heartily so that I get a bit sweaty and that I breathe properly so that I fill my lungs—so that I can be prepared—and be as strong as possible, in case I should catch that coronavirus” ([ 77 ], p. 9); see also [ 70 , 78 , 82 , 85 ].

Coping and adaptation

Many studies mentioned older adults’ processes of coping and adaptation during the pandemic [ 63 , 64 , 68 , 69 , 72 , 75 , 79 , 81 , 85 , 87 , 88 , 89 , 90 ].

A variety of coping processes were reported including: acceptance; behavioural adaptation; emotional regulation; creating new routines; or using new technology. Kremers et al. [ 79 ] reported: “We are very realistic about the situation and we all have to go through it. Better days will come” (p. e71). Behavioural adaptation was reported: “Because I’m asthmatic, I was wearing the disposable masks, I really had trouble breathing. But I was determined to find a mask I could wear” ([ 68 ], p. 14). New routines with protective hygiene helped some participants at the beginning of the pandemic to cope with the health threat: “I am washing my hands all the time, my hands are raw from washing them all the time, I don't think I need to wash them as much as I do but I do it just in case, I don’t have anybody coming in, so there is nobody contaminating me, but I keep washing” ([ 69 ], p. 4391); see also [ 72 ]. Verhage et al. [ 87 ] reported strategies of coping including self-enhancing comparisons, distraction, and temporary acceptance: “There are so many people in worse circumstances …” (p. e294). Other studies reported how participants used a new technology: “I have recently learned to use WhatsApp, where I can make video phone calls.” ([ 88 ], p. 163); see also [ 89 ].

Discrimination -intersectionality (age and race/gender identity)

Seven studies reported ageism, racism, and gender discrimination experienced by older adults during the pandemic [ 23 , 63 , 67 , 70 , 76 , 84 , 88 ].

Prigent et al. [ 84 ], conducted in a New Zealand study, found that ageism was reciprocal. Younger people spoke against older adults: “why don’t you do everyone a favour and drop dead you f******g b**** it’s all because of ones like you that people are losing jobs” (p. 11). On the other hand, older adults spoke against the younger generation: “Shame to see the much younger generations often flout the rules and generally risk the gains made by the team. Sheer arrogance on their part and no sanctions applied” (p.11). Although one study reported benevolent ageism [ 23 ] most studies found hostile ageism [ 23 , 63 , 67 , 70 , 76 , 84 ]. One study from Canada exploring 15 older adult’s Chinese immigrants’ experiences reported racism as people around them thought they would bring the virus into the country. The negative impact on existing friendships was told by a Chinese man aged 69 “I can tell some people are blatantly despising us. I can feel it. When I talked with my Caucasian friends verbally, they would indirectly blame us for the problem. Eventually, many of our friendships ended because of this issue” ([ 88 ], p161). In addition, this study reported ageism when participants in nursing homes felt neglected by the Canadian government.

Two papers reported experiences of sexual and gender minorities (SGM) (e.g. transgender, queer, lesbian or gay) and found additional burdens during the pandemic [ 63 , 76 ]. People experienced marginalisation, stereotypes, and discrimination, as well as financial crisis: “I have faced this throughout life. Now people look at me in a way as if I am responsible for the virus.” ([ 63 ], p. 6). The consequence of marginalisation and ignorance of people with different gender identities was also noted by Hafford- Letchfield et al. [ 76 ]: “People have been moved out of their accommodation into hotels with people they don't know …. a gay man committed suicide, community members know of several that have attempted suicide. They are feeling pretty marginalised and vulnerable and you see what people are writing on the chat pages” (p.4). The intersection of ageism, racism, and heterosexism and its negative impact on people’s well-being during the pandemic reflects additional burden and stressors for older adults.

This systematic literature review is important as it provides new insights into the lived experiences of older adults during the COVID-19 pandemic, worldwide. Our study highlights that the COVID-19 pandemic brought an increase in English-written qualitative articles to the fore. We found that 32 articles met the inclusion criteria but 5 were low quality. A lack of transparency reduces the trustworthiness of the study for the reader and the scientific community. This is particularly relevant as qualitative research is often criticised for its bias or lack of rigor [ 94 ]. However, their findings are additional evidence for our study.

Our aim was to explore, in a systematic literature review, the lived experiences of older adults during the COVID-19 pandemic worldwide. The evidence highlights the themes of desired and challenged wellbeing, coping and adaptation, and discrimination and intersectionality, on wellbeing.

Perceived risk communication was experienced by many participants as overwhelming and anxiety-provoking. This finding supports Anwar et al.’s [ 37 ] study from the beginning of the pandemic which found, in addition to circulating information, that mass media influenced the public's behaviour and in consequence the spread of disease. The impact can be positive but has also been revealed to be negative as well. They suggest evaluating the role of the mass media in relation to what and how it has been conveyed and perceived. The disrupted social connectivity found in our review supports earlier studies that reported the negative impact of people’s well-being [ 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 ] at the beginning of the pandemic. This finding is important for future health crisis management, as the protective health measures such as confinement or self-isolation had a negative impact on many of the participants’ emotional wellbeing including increased anxiety, feelings of depression, and loneliness during the lockdowns. As a result of our review, future protective health measures should support people’s desire to maintain proximity with their loved ones and friends. However, we want to stress that our findings are mixed.

The ability of older adults to adapt and cope with the health crisis is important: many of the reported studies noted the diverse strategies used by older people to adapt to new circumstances. These included learning new technologies or changing daily routines. Politicians and the media and politicians should recognise both older adults' risk of disease and its consequences, but also their adaptability in the face of fast-changing health measures. This analysis supports studies conducted over the past decades on lifespan development, which found that people learn and adapt livelong to changing circumstances [ 95 , 96 , 97 ].

We found that discrimination against age, race, and gender identity was reported in some studies, in particular exploring participants’ experiences with immigration backgrounds and sexual and gender minorities. These studies highlighted the intersection of age and gender or race and were additional stressors for older adults and support the findings from Ramirez et al. [ 98 ] This review suggests that more research should be conducted to investigate the experiences of minority groups to develop relevant policies for future health crises.

Our review was undertaken two years after the pandemic started. At the cut-off point of our search strategy, no longitudinal studies had been found. However, in December 2022 a longitudinal study conducted in the USA explored older adult’s advice given to others [ 99 ]. They found that fostering and maintaining well-being, having a positive life perspective, and being connected to others were coping strategies during the pandemic [ 100 ]. This study supports the results of the higher order constructs of coping and adaptation in this study. Thus, more longitudinal studies are needed to enhance our understanding of the long-term consequences of the COVID-19 pandemic. The impact of the COVID-19 restrictions on older adults’ lives is evident. We suggest that future strategies and policies, which aim to protect older adults, should not only focus on the physical health threat but also acknowledge older adults' needs including psychological support, social connectedness, and instrumental support. The policies regarding older adult’s protections changed quickly but little is known about older adults’ involvement in decision making [ 100 ]. We suggest including older adults as consultants in policymaking decisions to ensure that their own self-determinism and independence are taken into consideration.

There are some limitations to this study. It did not include the lived experiences of older adults in care facilities or hospitals. The studies were undertaken during the COVID-19 pandemic and therefore data collection was not generally undertaken face-to-face. Thus, many studies included participants who had access to a phone, internet, or email, others could not be contacted. Additionally, we did not include published papers after August 2022. Even after capturing the most commonly used terms and performing additional hand searches, the search terms used might not be comprehensive. The authors found the quality of the papers to be variable, and their credibility was in question. We acknowledge that more qualitative studies might have been published in other languages than English and were not considered in this analysis.

To conclude, this systematic literature review found many similarities in the experiences of older adults during the Covid-19 pandemic despite cultural and socio-economic differences. However, we stress to acknowledge the heterogeneity of the experiences. This study highlights that the interplay of mass media reports of the COVID-19 pandemic and the policies to protect older adults had a direct impact on older adults’ well-being. The intersection of ‘isms’ (ageism, racism, and heterosexism) brought an additional burden for some older adults [ 98 ]. These results and knowledge about the drawbacks of health-protecting measures need to be included in future policies to maintain older adults’ well-being during a health crisis.

Availability of data and materials

The systematic literature review is based on already published articles. And all data analysed during this study are included in this manuscript. No additional data was used.

World Health Organisation (WHO): WHO Director-General’s opening remarks at the media briefing on COVID-19 - 11 March 2020. 2020. Retrieved from https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020.

WorldOMeter: Coronavirus toll update: Cases & deaths by country. 2022. Worldometers. https://www.worldometers.info/coronavirus/ .

World Health Organisation (WHO): Coronavirus disease. 2022. https://www.who.int/health-topics/coronavirus#tab=tab . Aaccessed 5 Oct 2022.

Ayouni I, Maatoug J, Dhouib W, Zammit N, Fredj SB, Ghammam R, Ghannem H. Effective public health measures to mitigate the spread of COVID-19: a systematic review. BMC Public Health. 2021;21(1):1015.

PubMed   PubMed Central   CAS   Google Scholar  

Cerasoli B. The frail future of geriatrics. Lancet Healthy Longev. 2020;1(1):e11.

PubMed   PubMed Central   Google Scholar  

Allen K-A, Arslan G, Craig H, Arefi S, Yaghoobzadeh A, Sharif Nia H. The psychometric evaluation of the sense of belonging instrument (SOBI) with Iranian older adults. BMC Geriatr. 2021;21(1):211.

Bailey L, Ward M, DiCosimo A, Baunta S, Cunningham C, Romero-Ortuno R, Kenny RA, Purcell R, Lannon R, McCarroll K, et al. Physical and mental health of older people while cocooning during the COVID-19 pandemic. QJM. 2021;114(9):648–53.

Dahlberg L. Loneliness during the COVID-19 pandemic. Aging Ment Health. 2021;25(7):1161–4.

PubMed   Google Scholar  

Derrer-Merk E, Ferson S, Mannis A, Bentall R, Bennett KM. Older people's family relationships in disequilibrium during the COVID-19 pandemic. What really matters? Ageing Soc. 2022:1–18.

Derrer-Merk E, Ferson S, Mannis A, Bentall RP, Bennett KM. Belongingness challenged: Exploring the impact on older adults during the COVID-19 pandemic. PLoS ONE. 2022;17(10):e0276561.

Heidinger T, Richter L. The effect of COVID-19 on loneliness in the elderly. An empirical comparison of pre-and peri-pandemic loneliness in community-dwelling elderly. Front Psychol. 2020;11:(585308).

Richter L, Heidinger T. Hitting close to home: the effect of COVID-19 illness in the social environment on psychological burden in older adults. Front Psychol. 2021;12:(737787).

Krendl AC, Perry BL. The impact of sheltering in place during the COVID-19 pandemic on older adults’ social and mental well-being. J Gerontol B Psychol Sci Soc Sci. 2021;76(2):e53–8.

Litwin, H., & Levinsky, M. Social networks and mental health change in older adults after the Covid-19 outbreak. Aging Ment Health. 2021:1–6. https://doi.org/10.1080/13607863.2021.1902468 .

De Pue S, Gillebert C, Dierckx E, Vanderhasselt MA, De Raedt R, Van den Bussche E. The impact of the COVID-19 pandemic on wellbeing and cognitive functioning of older adults. Sci Rep. 2021;11(1):4636.

Shahid Z, Kalayanamitra R, McClafferty B, Kepko D, Ramgobin D, Patel R, Aggarwal CS, Vunnam R, Sahu N, Bhatt D, et al. COVID-19 and older adults: what we know. J Am Geriatr Soc. 2020;68(5):926–9.

Smith M, Steinman L, Casey E. Combatting social isolation among older adults in a time of physical distancing: the COVID-19 social connectivity paradox. Front Public Health. 2020;8:403.

Cohn-Schwartz E, Finlay JM, Kobayashi LC. Perceptions of societal ageism and declines in subjective memory during the COVID-19 pandemic: longitudinal evidence from US adults aged ≥55 years. J Soc Issues. 2022;78(4):924–38.

Tsoukalis-Chaikalis N, Demsia S, Stamatopoulou A, Chaniotis D, Tel Nu. Systematic Mapping Study of Covid-19 Psychological Impact. 2021;15(3):820.

van Gerwen M, Alsen M, Little C, Barlow J, Genden E, Naymagon L, Tremblay D. Risk factors and outcomes of COVID-19 in New York City; a retrospective cohort study. J Med Virol. 2021;93(2):907–15.

Victor CR, Rippon I, Barreto M, Hammond C, Qualter P. Older adults’ experiences of loneliness over the lifecourse: an exploratory study using the BBC loneliness experiment. Arch Gerontol Geriatr. 2022;102:104740.

Vrach IT, Tomar R. Mental health impacts of social isolation in older people during COVID pandemic. Prog Neurol Psychiatry. 2020;24(4):25–9.

Google Scholar  

Derrer-Merk E, Reyes-Rodriguez M-F, Salazar A-M, Guevara M, Rodríguez G, Fonseca A-M, Camacho N, Ferson S, Mannis A, Bentall RP, et al. Is protecting older adults from COVID-19 Ageism? A comparative cross-cultural constructive grounded theory from the United Kingdom and Colombia. J Soc Issues. 2022;78(4):900–23.

Lytle A, Levy SR. Reducing ageism toward older adults and highlighting older adults as contributors during the COVID-19 pandemic. J Soc Issues. 2022;78(4):1066–84.

McDarby M, Ju CH, Picchiello MC, Carpenter BD. Older adults’ perceptions and experiences of ageism during the COVID-19 pandemic. J Soc Issues. 2022;78(4):939–64.

Drury L, Abrams D, Swift HJ. Intergenerational contact during and beyond COVID-19. J Soc Issues. 2022;78(4):860–82.

Kanık B, Uluğ ÖM, Solak N, Chayinska M. “Let the strongest survive”: Ageism and social Darwinism as barriers to supporting policies to benefit older individuals. J Soc Issues. 2022;00:1–25.

Spaccatini F, Giovannelli I, Pacilli MG. “You are stealing our present”: Younger people's ageism towards older people predicts attitude towards age-based COVID-19 restriction measures. J Soc Issues. 2022;78(4):769–89.

Sutter A, Vaswani M, Denice P, Choi KH, Bouchard H, Esses VM. Ageism toward older adults during the COVID-19 pandemic: Intergenerational conflict and support. J Soc Issues. 2022;78(4):815–41.

Swift H, Chasteen A. Ageism in the time of COVID-19. Group Process Intergroup Relat. 2021;24:246–52.

World Health Organisation (WHO): Risk Communication in times of an epidemic or pandemic. Asset paper series Epidemics and Pandemics: 2017. https://www.who.int/emergencies/risk-communicationsWHORCaCERaoSAo .

Bailey A, Harris MA, Bogle D, Jama A, Muir SA, Miller S, Walters CA, Govia I. Coping with COVID-19: health risk communication and vulnerable groups. Disaster Med Public Health Prep. 2021;17:e22.

Li X, Lyu H. Epidemic risk perception PS, and mental health during COVID-19 pandemic: a moderated mediating model. Front Psychol. 2021;11:563741. https://doi.org/10.3389/fpsyg.2020.563741 .

Article   PubMed   PubMed Central   Google Scholar  

Sandman P M & American Industrial Hygiene Association: Responding to community outrage: strategies for effective risk communication. American Industrial Hygiene Association. 2003.

Sandman, P M. Introduction and orientation index.  https://www.psandman.com/index-intro.htm . Retrieved 13 October 2022.

Anderson M, Pitchforth E, Asaria M, Brayne C, Casadei B, Charlesworth A, Coulter A, Franklin BD, Donaldson C, Drummond M, et al. LSE-Lancet Commission on the future of the NHS: re-laying the foundations for an equitable and efficient health and care service after COVID-19. Lancet. 2021;397(10288):1915–78.

PubMed   CAS   Google Scholar  

Anwar A, Malik M, Raees V, Anwar A. Role of mass media and public health communications in the COVID-19 pandemic. Cureus. 2020;12(9):e10453.

U.S. National Library of Medicine: COVID-19 - Search Details - ClinicalTrials.gov: Clinicaltrials.gov 2022. Retrieved October 13, 2022, from https://clinicaltrials.gov/ct2/results/details?cond=COVID-19 .

Tremblay S, Castiglione S, Audet L-A, Desmarais M, Horace M, Peláez S. Conducting qualitative research to respond to COVID-19 challenges: reflections for the present and beyond. Int J Qual Methods. 2021;20:16094069211009680.

Flemming K, Noyes J. Qualitative evidence synthesis: where are we at? Int J Qual Methods. 2021;20:1609406921993276.

Booth A. Searching for qualitative research for inclusion in systematic reviews: a structured methodological review. Syst Rev. 2016;5(1):74.

Krnic Martinic M, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203.

Phillips V, Barker E. Systematic reviews: structure, form and content. J Perioper Pract. 2021;31(9):349–53.

Derrer-Merk E, Soulsby L, Reyes Rodriguez M, Ferson S, Mannis A, Bennett KM. Older adults' lived experiences during the COVID-19 pandemic: a systematic review. 2022. PROSPERO 2022 CRD42022331714. Available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022331714 .

Cooke A, Smith D, Booth A. Beyond PICO: the SPIDER tool for qualitative evidence synthesis. Qual Health Res. 2012;22:1435–43.

Long HA, French DP, Brooks JM. Optimising the value of the critical appraisal skills programme (CASP) tool for quality appraisal in qualitative evidence synthesis. Res Methods Med Health Sci. 2020;1(1):31–42.

Sattar R, Lawton R, Panagioti M, Johnson J. Meta-ethnography in healthcare research: a guide to using a meta-ethnographic approach for literature synthesis. BMC Health Serv Res. 2021;21(1):50.

Soundy A, Heneghan NR. Meta-ethnography. Int Rev Sport Exerc Psychol. 2022;15(1):266–86.

Fallon V, Groves R, Halford JCG, Bennett KM, Harrold JA. Postpartum anxiety and infant-feeding outcomes: a systematic review. J Hum Lact. 2016;32(4):740–58.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583.

Tod D, Booth A, Smith B. Critical appraisal. Int Rev Sport Exerc Psychol. 2022;15(1):52–72.

Morse J. Why the Qualitative Health Research (QHR) review process does not use checklists. Qual Health Res. 2021;31(5):819–21.

Munthe-Kaas HM, Glenton C, Booth A, Noyes J, Lewin S. Systematic mapping of existing tools to appraise methodological strengths and limitations of qualitative research: first stage in the development of the CAMELOT tool. BMC Med Res Methodol. 2019;19(1):113.

Njau B, Covin C, Lisasi E, Damian D, Mushi D, Boulle A, Mathews C. A systematic review of qualitative evidence on factors enabling and deterring uptake of HIV self-testing in Africa. BMC Public Health. 2019;19(1):1289.

Butler A, Hall H, Copnell B. A guide to writing a qualitative systematic review protocol to enhance evidence-based practice in nursing and health care. Worldviews Evid Based Nurs. 2016;13(3):241–9.

Soilemezi D, Linceviciute S. Synthesizing qualitative research: reflections and lessons learnt by two new reviewers. Int J Qual Methods. 2018;17(1):1609406918768014.

Booth A. Harnessing energies, resolving tensions: acknowledging a dual heritage for qualitative evidence synthesis. Qual Health Res. 2019;29(1):18–31.

Carroll C, Booth A. Quality assessment of qualitative evidence for systematic review and synthesis: Is it meaningful, and if so, how should it be performed? Res Synth Methods. 2015;6(2):149–54.

Charmaz K. Constructing Grounded Theory. 2nd ed. London: Sage; 2014.

Toye F, Seers K, Allcock N, Briggs M, Carr E, Barker K. Meta-ethnography 25 years on: challenges and insights for synthesising a large number of qualitative studies. BMC Med Res Methodol. 2014;14(1):80.

Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. California: Sage Publications Ltd; 1988.

Akkus Y, Parlak A, Karacan Y, Karatay G. Perceptions and experiences of older people regarding the COVID-19 pandemic process: a phenomenological study. Turk Geriatri Dergisi. 2021;24:546–56.

Banerjee D, Rao TSS. “The Graying Minority”: lived experiences and psychosocial challenges of older transgender adults during the COVID-19 pandemic in India, A qualitative exploration. Front Psychiatry. 2021;11:604472.

Brooke J, Jackson D. Older people and COVID-19: Isolation, risk and ageism. J Clin Nurs. 2020;29(13-14):2044–6.

Bundy H, Lee HM, Sturkey KN, Caprio AJ. The lived experience of already-lonely older adults during COVID-19. Gerontologist. 2021;61(6):870–7.

Chemen S, Gopalla YN. Lived experiences of older adults living in the community during the COVID-19 lockdown - The case of mauritius. J Aging Stud. 2021;57:100932.

Falvo I, Zufferey MC, Albanese E, Fadda M. Lived experiences of older adults during the first COVID-19 lockdown: a qualitative study. PLoS ONE. 2021;16(6):e0252101.

Fiocco AJ, Gryspeerdt C, Franco G. Stress and adjustment during the COVID-19 pandemic: a qualitative study on the lived experience of Canadian older adults. Int J Environ Res Public Health. 2021;18(24):12922.

Fristedt S, Carlsson G, Kylén M, Jonsson O, Granbom M. Changes in daily life and wellbeing in adults, 70 years and older, in the beginning of the COVID-19 pandemic. Scand J Occup Ther. 2022;29(6):511–21.

Gazibara T, Maksimovic N, Dotlic J, Jeremic Stojkovic V, Cvjetkovic S, Milic M. Experiences and aftermath of the COVID-19 lockdown among community-dwelling older people in Serbia: a qualitative study. J Eval Clin Pract. 2022;28(4):631–40.

Giebel C, Ivan B, Ddumba I. COVID-19 public health restrictions and older adults’ well-being in Uganda: psychological impacts and coping mechanisms. Clin Gerontol. 2022;45(1):97–105.

Goins RT, Anderson E, Minick H, Daniels H. Older adults in the United States and COVID-19: a qualitative study of perceptions, finances, coping, and emotions. Front Public Health. 2021;9:660536.

Gomes MACFC, Fontenele NAO, Galindo Neto NM, Barros LM, Frota NM. Elderly people’s experience facing social isolation in the COVID-19 pandemic. Rev Rene. 2021;22:e69236. https://doi.org/10.15253/2175-6783.20212269236 .

Article   Google Scholar  

Gonçalves AR, Barcelos JLM, Duarte AP, Lucchetti G, Gonçalves DR. Silva e Dutra FCM, Gonçalves JRL: Perceptions, feelings, and the routine of older adults during the isolation period caused by the COVID-19 pandemic: a qualitative study in four countries. Aging Ment Health. 2022;26(5):911–8.

Greenwood-Hickman MA, Dahlquist J, Cooper J, Holden E, McClure JB, Mettert KD, Perry SR, Rosenberg DE. “They’re Going to Zoom It”: a qualitative investigation of impacts and coping strategies during the COVID-19 pandemic among older adults. Front Public Health. 2021;9:679976.

Hafford-Letchfield T, Toze M, Westwood S. Unheard voices: a qualitative study of LGBT+ older people experiences during the first wave of the COVID-19 pandemic in the UK. Health Soc Care Community. 2022;30(4):e1233–43.

Huntley R, Bratt AS. An interpretative phenomenological analysis of the lived experiences of older adults during the covid-19 pandemic in sweden. Nordic Psychol. 2022;75(1)3–19.

Jiménez-Etxebarria E, Bernaras Iturrioz E, Jaureguizar J. Impact of the COVID-19 pandemic as perceived by older people in Northern Spain. Psychol Res Behav Manag. 2021;14:1789–803.

Kremers EM, Janssen JHM, Nieuwboer MS, Olde Rikkert MGM, Peeters GMEE. The psychosocial adaptability of independently living older adults to COVID-19 related social isolation in the Netherlands: a qualitative study. Health Soc Care Community. 2022;30(1):e67–74.

Kulmala J, Tiilikainen E, Lisko I, Ngandu T, Kivipelto M, Solomon A. Personal social networks of community-dwelling oldest old during the Covid-19 pandemic—A qualitative study. Front Public Health. 2021;9:1–10.

Mahapatra P, Sahoo KC, Desaraju S, Pati S. Coping with COVID-19 pandemic: reflections of older couples living alone in urban Odisha India. Prim Health Care Res Dev. 2021;22:e64.

McKinlay AR, Fancourt D, Burton A. A qualitative study about the mental health and wellbeing of older adults in the UK during the COVID-19 pandemic. BMC Geriatr. 2021;21(1):1–10.

Pfabigan J, Wosko P, Pichler B, Reitinger E, Pleschberger S. Under reconstruction: the impact of COVID-19 policies on the lives and support networks of older people living alone. Int J Care Caring. 2022;6(1-2):211–28.

Prigent C, Morgan T, Wiles J, Morgan K, Williams L, Gott M. Intergenerational tension or cohesion during the covid-19 pandemic?: A letter-writing study with older new zealanders. J Intergenerational Relationsh. 2022;20(4):386–405.

Sangrar R, Porter MM, Chesser S. Exploring the interpretation of COVID-19 messaging on older adults. Experiences of vulnerability. Can J Aging Rev Can Vieil. 2021;40(4):533–42.

Sattari S, Billore S. Bring it on Covid-19: being an older person in developing countries during a pandemic. Working Older People. 2020;24(4):281–91.

Verhage M, Thielman L, Kock L, Lindenberg J. Coping of older adults in times of COVID-19: considerations of temporality among dutch older adults. J Gerontol B. 2021;76(7):e290–9.

Wang Q, Liu JKK, Walsh CA. Identities: experiences and impacts of the COVID-19 pandemic from the perspectives of older Chinese immigrants in Canada. China J Soc Work. 2021;14(2):153–71.

Xie B, Shiroma K, De Main AS, Davis NW, Fingerman K, Danesh V. Living through the COVID-19 pandemic: community-dwelling older adults’ experiences. J Aging Soc Policy. 2021;33(4–5):380–97.

Yang Q, Wang Y, Tian C, Chen Y, Mao J. The Experiences of Community-dwelling older adults during the COVID-19 lockdown in Wuhan: a qualitative study. J Adv Nurs (John Wiley & Sons, Inc). 2021;77(12):4805–14.

Yıldırım H. Psychosocial status of older adults aged 65 years and over during lockdown in Turkey and their perspectives on the outbreak. Health Soc Care Community. 2022;30(3):899–907.

Hill Z, Tawiah-Agyemang C, Kirkwood B, Kendall C. Are verbatim transcripts necessary in applied qualitative research: experiences from two community-based intervention trials in Ghana. Emerg Themes Epidemiol. 2022;19(1):5.

Giles D. Performed across Europe and written up in English: the double challenge for qualitative psychologists. Qual Res Psychol. 2019;16(3):503–7.

Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010;74(8):141.

Nikitin J, Freund A. The Adaptation Process of Aging. The Cambridge Handbook of Successful Aging. Chapter: 17. Cambridge University Press. 2019. p. 281–98.

Deimling GT, Kahana B, Bowman KF, Schaefer ML. Cancer survivorship and psychological distress in later life. Psychooncology. 2002;11(6):479–94.

Baltes PB, Staudinger UM, Lindenberger U. Lifespan psychology: theory and application to intellectual functioning. Annu Rev Psychol. 1999;50:471–507.

Ramirez L, Monahan C, Palacios-Espinosa X, Levy SR. Intersections of ageism toward older adults and other isms during the COVID-19 pandemic. J Soc Issues. 2022;78(4):965–90.

Van Vleet B, Fuller HR, Hofmann B, Huseth-Zosel A. Untapped insight: a longitudinal qualitative analysis of older adults’ advice during the COVID-19 pandemic. Innov Aging. 2022;6(7):igac071.

Gietel-Basten S, Matus K, Mori R. COVID-19 as a trigger for innovation in policy action for older persons? Evid Asia Policy Soc. 2022;41(1):168–86.

Download references

Acknowledgements

Not applicable

Not applicable.

Author information

Authors and affiliations.

University of Liverpool, Eleanor Rathbone Building, Bedford Street South, Liverpool, L697ZA, UK

Elfriede Derrer-Merk, Laura K. Soulsby & Kate M. Bennett

Universidad de los Andes, Carrera 1 No. 18A-12, Bogotá, Colombia

Maria-Fernanda Reyes-Rodriguez

Principal Health Psychologist, Resilience Hub, Lancashire & South Cumbria Foundation Hospital, Chorley, UK

Louise Roper

You can also search for this author in PubMed   Google Scholar

Contributions

Elfriede Derrer-Merk contributed to the design, analysis, and writing the draft. Maria-Fernanda Rodriguez-Reyes contributed to the analysis, revised the draft, and approved the submission. Laura K. Soulsby contributed to the analysis, revised the draft, and approved the submission. Louise Roper contributed to the analysis, revised the draft, and approved the submission. Kate M. Bennett contributed to the design, analysis, writing the draft, and approved the submission.

Corresponding author

Correspondence to Elfriede Derrer-Merk .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Derrer-Merk, E., Reyes-Rodriguez, MF., Soulsby, L.K. et al. Older adults’ experiences during the COVID-19 pandemic: a qualitative systematic literature review. BMC Geriatr 23 , 580 (2023). https://doi.org/10.1186/s12877-023-04282-6

Download citation

Received : 16 November 2022

Accepted : 05 September 2023

Published : 20 September 2023

DOI : https://doi.org/10.1186/s12877-023-04282-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Older adults
• Risk communication
• Discrimination

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Review Article
• Open access
• Published: 02 April 2024

How do we study misogyny in the digital age? A systematic literature review using a computational linguistic approach

• Lara Fontanella   ORCID: orcid.org/0000-0002-5441-0035 1 ,
• Berta Chulvi 2 , 3 ,
• Elisa Ignazzi 4 ,
• Annalina Sarra 5 &
• Alice Tontodimamma 1  

Humanities and Social Sciences Communications volume  11 , Article number:  478 ( 2024 ) Cite this article

Metrics details

• Cultural and media studies

Nowadays, despite centuries of striving for equality, women still face higher levels of discrimination compared to men in nearly every aspect of life. Recently, this systemic inequality has manifested in cyberspace through the proliferation of abusive content that is even more aggressive than what one would expect in the 21st century. Various research disciplines are now attempting to characterise this new manifestation of misogyny. The endeavour to comprehend this phenomenon has resulted in a significant increase in publications from several fields, including Social Sciences, Arts and Humanities, Psychology, and Computer Science. This paper presents a systematic review of multidisciplinary research on misogyny from the years 1990 to 2022, encompassing a total of 2830 articles retrieved from the Scopus database as of December 31, 2022. The literature is thoroughly analysed using three approaches: bibliometric analysis, topic detection, and qualitative analysis of the documents. The findings suggest that the analysis of online misogyny has been the primary driver behind the exponential growth in publications in this field. Additionally, the results of the topic analysis and topic interaction reveal a limited connection between the areas of knowledge that are necessary to fully grasp this complex phenomenon.

Similar content being viewed by others

The old-new epistemology of digital journalism: how algorithms and filter bubbles are (re)creating modern metanarratives

Luca Serafini

The dark web privacy dilemma: linguistic diversity, talkativeness, and user engagement on the cryptomarket forums

Zhicong Chen, Xiang Meng & Cheng-Jun Wang

Community and authority in ROAR Magazine

Introduction.

Nowadays, regardless of centuries of fighting for equality, women continue to face a disproportionate amount of discrimination compared to men across various contexts. Women and girls encounter prejudice, sexist attitudes, open discrimination, and violence throughout their lives, while the extent of these experiences varies by location, identity, and culture. Disgust, intolerance, or entrenched prejudice, serving to legitimise women’s oppression, persist even in countries often alleged to be post-patriarchal, like the United States, Australia, and the United Kingdom (Manne, 2017 ). The all-pervasive form of hostility and aversion against women and girls is referred to as misogyny, a term derived from the Ancient Greek word “mĩsoguniã”, which means hatred of women. According to Allen ( 2021 ), misogyny has a disputed definition. Some authors offer a definition of misogyny that, in some respects, overlaps with the concept of sexism. For example, Code ( 2000 ) defines misogyny as any of the following acts or feelings: sexual and physical violence against women, exclusion of women, promotion of patriarchy, belittlement, and marginalisation of women. In this approach, the promotion of patriarchy, broadly conceptualised as a system or systems producing and reproducing gendered and intersectional inequalities, is clearly the spread of a sexist mentality. Here, sexism is linked to the acceptance of sex-role stereotypes and can manifest at various levels: individual, organisational, institutional, and cultural (VandenBos, 2015 ). In the same line of reasoning, Jukes ( 1993 ) states that misogyny can be obvious and explicit at times, but it can also be subtle and insidious. However, the subtle expression of misogyny is more linked to sexist attitudes than to the expression of hate. Other authors, such as Manne ( 2017 ), set out a clear distinction between sexism and misogyny. In her book, “Down Girl: The Logic of Misogyny”, Kate Manne ( 2017 ) describes misogyny as the patriarchal order’s “law enforcement” branch, which rewards “good” women who adhere to social norms while punishing those who disobey. Sexism, on the other hand, is viewed as the “justificatory” branch, which rationalises and justifies male dominance through beliefs, theories, stereotypes, and cultural narratives that portray women as naturally inferior. This conceptual debate is due to two reasons. First, the fact that misogyny is strictly linked to the concepts of patriarchy and sexism, and second, the evidence that our societies are facing new ways of conveying misogynistic content in the form of open denigration of women.

Focusing on the link between concepts that describe women’s discrimination, it is evident that the powerful dynamics of a patriarchal society contribute to the development of a sexist culture, and this leads to the oppression of women both in their personal lives and within societal institutions (Millet, 1970 ). Additionally, hostile and benevolent sexism (Glick and Fiske, 1997 ) functions to preserve patriarchy and conventional gender norms. Benevolent sexism manifests through subjectively positive attitudes towards women in traditional roles, encompassing protective paternalism, idealisation, and a desire for intimacy. On the other hand, hostile sexism is expressed in a blatant and resentful way toward women who violate traditional roles and includes the negative equivalents of each dimension of benevolent sexism: dominant paternalism, derogatory beliefs, and heterosexual hostility. The aforementioned patriarchal culture legitimises openly misogynistic expressions, which represent the most extreme manifestation of aggression against women.

In this complex dynamic, studies from different disciplines tend to use different terminology when examining hostility towards women. Specifically, research in psychology is more inclined to use terms related to sexism, especially in distinguishing between hostile and benevolent sexism, and the notion of patriarchy is extensively examined in social science, particularly in sociological studies. The concept of misogyny is more commonly used in communication studies and computational science. The findings reported in the Supplementary Material provide evidence of the emphasis of different disciplines on different concepts.

Regarding the emergence of new ways of transmitting misogynistic content, the rise of interactive social media has been widely considered (Moloney and Love, 2018 ; Rubio Martìn and Gordo Lòpez, 2021 ; Tranchese and Sugiura, 2021 ).

Misogyny on the internet is not a new phenomenon. Indeed, legislation pertaining to women’s online safety dates back to the Beijing Declaration in 1995. However, it was not until the events of Gamergate Footnote 1 (Massanari, 2020 ) in August 2014 that the mainstream media and academic research took notice. In fact, in the gaming community, 2014 saw the emergence of the controversy and online movement known as “Gamergate”. It started out as a reaction to questions about ethics in video game journalism, but it soon turned into a harassment campaign directed at female journalists. The movement brought attention to misogyny, sexism, and the need for diversity in the gaming industry.

With the development of social networks, the historical aversion to women has become articulated through new modes of communication and social interaction. While digital spaces have amplified female voices, online platforms have notoriously facilitated the spread of misogynistic content: women’s systematic inequality and discrimination have been replicated in cyberspace in the form of abusive content much more aggressive than we would have expected in the 21st century (Bates, 2021 ). The online realm provides ample opportunities for misogyny to be linguistically expressed in various ways, ranging from subtle forms such as social exclusion and discrimination to more severe forms like sexual objectification and violent threats (Anzovino et al., 2018 ). Studies examining online misogynistic discourse have employed different terminology, such as “gender cyber hatred” (Jane, 2017 ), “cyber harassment” (Citron, 2014 ), “technological violence” (Ostini and Hopkins, 2015 ), “gender trolling” (Mantilla, 2013 ), “e-bile”, and “gender hate speech” (Jane, 2015 ). Other scholars (see, for instance, Ging and Siapera, 2018 ) chose to use a broader definition of misogyny which almost always results in some form of harm, either directly, in the form of psychological, professional, or physical harm, or indirectly, making the internet a less equal, less safe, or less inclusive space for women and girls.

Our study aims to investigate the current state of research on misogyny. For this purpose, we focus on the scientific literature on this subject during the period between 1990 and 2022. To the best of our knowledge, our study is the first systematic review on misogyny which combines three approaches: bibliometrics, topic detection, and qualitative analysis of the documents.

For the bibliometric research, we first analyse the existing literature extracted from the Scopus database within the misogyny research field by exploiting bibliometric tools. Bibliometric analysis provides a systematic, transparent, and replicable manner to investigate extant literature in a given field and discover the progress of disciplinary research from a macro perspective, supporting future research directions. Using bibliometric methods, we explore the main lines of research in the scientific literature on misogyny and offer a summary of the research activity in terms of the volume of work and evolution over time, as well as in terms of the social, intellectual and conceptual structures of this research area.

Although bibliometric tools provide a broad overview of current research, they cannot deliver detailed insights into studies in the literature based on semantic content analysis. In order to conduct an in-depth semantic analysis, it is necessary to supplement bibliometric methods with text-mining techniques (Hu et al., 2014 ). In accordance, our work employs topic analysis based on the Latent Dirichlet Allocation method (LDA; Blei et al., 2003 ) in order to identify the most prevalent latent themes in misogyny literature. LDA is gaining popularity among scholars in diverse fields (Alghamdi and Alfalqi, 2015 ). Two important findings emerge from a topic model: a list of topics (i.e., clusters of words that appear frequently together) and a list of documents that are strongly associated with each topic. As a result, this method offers a probabilistic quantification of relevance for both the identification of topics and the classification of documents, making it useful for locating interpretable topics with semantic meaning and assigning these topics to literature documents (Tontodimamma et al., 2021 ). According to Suominen and Toivanen ( 2016 ), the main innovation of topic modelling in categorising scientific knowledge is that it essentially eliminates the need to fit knowledge that is brand-new to the world into definitions that are already well-established.

Finally, we complement the study with a qualitative analysis aimed to discover the sociological perspective of the literature on online misogyny, on the one hand, and the computational aspects, on the other hand.

Bibliometric analyses

Bibliographic dataset.

For the analysis, we use a bibliometric dataset covering the period 1990–2022, retrieved from the Scopus database on 31 December 2022. Since we focus on the broad spectrum of scientific research on misogyny, the bibliographic dataset was extracted by looking for publications containing terms related to the generic query “misogyn*” in the content of the title, abstract, and keywords. All types of publications were included in the search, and 2830 documents were retrieved. The top publication fields include Social Sciences, Arts and Humanities, Psychology, and Computer Science.

Information about document distribution by research field is given in the Supplementary Material , along with the document distribution by source and the ranking of the most productive countries and authors.

Research activity

The evolution over time of the number of published documents shows remarkable growth (see Fig. 1 ). We found out that the number of published documents has increased dramatically over time. Since 1992, it has been possible to distinguish two distinct phases. A gradual increase in publications occurred during the first phase, which lasted from 1990 to 2010. The second phase, from 2010 to 2022, has a higher growth rate, indicating increased interest. This finding aligns with the three-stage development theory (Price, 1963 ) of productivity on a particular subject. Small increments in the scientific literature are documented during the precursor period when some scholars begin publishing research on a new topic. The number of papers increases exponentially in the second phase as the topic expands and draws a growing number of scientists, as many facets of the subject remain unexplored. Finally, in the third phase, the curve aspect shifts from exponential to logistic, testifying to a stabilisation in production and a consolidation of the body of knowledge.

Number of publications on misogyny per year: observed and expected temporal evolution according to exponential growth.

To verify the rapid increase, we fit an exponential growth curve to the data. The yearly rate of change in this model is 13.1%, demonstrating how research on misogyny might be cast in the second phase of development: although more research is being released, there is still space for improvement in many areas.

It is noteworthy to highlight that, as shown in the Supplementary Material , the research on misogyny from 1990 to 2002 follows a similar trend as sexism and has a slightly higher yearly growth rate compared to patriarchy. However, when considering only the five years prior to 2022, a more noticeable rise in the volume of published research on misogyny becomes evident, with a twofold increase in the number of published documents.

Social Structure of research on misogyny: collaboration network

To capture the essential characteristics of the misogyny research field, with a specific emphasis on collaborative efforts among different authors, we construct the authors’ collaboration network. We used the Bibliometrix R package, for performing network analysis and visualisation (Aria and Cuccurullo, 2017 ). Within the collaboration network, researchers act as nodes, and the connections between them (edges) represent co-authorships on articles. The node size is indicative of the authors’ productivity, measured in terms of the number of manuscripts authored or co-authored. The edges are weighted according to the frequency of co-authorship. Figure 2 visually illustrates the collaboration network among authors, highlighting the most significant cliques, each distinguished by different colours. The term “clique” is commonly employed to identify highly interconnected groups of elements, such as nodes or vertices, within a network. In our context, a “clique” signifies a group of authors who closely and frequently collaborate with one another compared to their counterparts, thereby creating a densely interconnected structure within the network. The most central scholars, with the highest number of connections, are Elisabetta Fersini, Paolo Rosso, Bilal Ghanem and Viviana Patti, who are also among the most proficient authors in the field of research on misogyny, as shown in the Supplementary Material . The noteworthy aspect is that the densest subgraphs link authors whose research falls under the computer science category.

Authors’ collaboration network.

Intellectual Structure of research on misogyny: citation analysis

The top five documents with the highest number of citations are: “Gamergate and The Fappening: How Reddit’s algorithm, governance, and culture support toxic technocultures” (Massanari, 2017 ), “Down girl: The logic of misogyny” (Manne, 2017 ), “Attitudinal antecedents of rape myth acceptance: A theoretical and empirical re-examination” (Lonsway and Fitzgerald, 1995 ), “Post-postfeminism?: new feminist visibilities in postfeminist times” (Gill, 2016 ) and “Beauty and Misogyny Harmful Cultural Practices in the West” (Jeffreys, 2005 ). These works investigate misogyny from various angles.

Manne’s book explores the logic of misogyny, which “primarily targets women because they are women in a man’s world ” (Manne, 2017 , p. 64). Manne argues that misogyny still exists in alleged post-patriarchal cultures and has taken different forms since legal equality, requiring women to be moral “givers” and validating a sense of entitlement among privileged men. Misogyny often takes the form of taking from women what they supposedly owe men and preventing women from competing for positions of power and authority traditionally held by men. In addition, Manne examines various examples of rape culture, including online harassment.

Considering attitudes toward sexual violence, Lonsway and Fitzgerald investigate the relationship between misogyny and rape myth acceptance. Here, rape myths can be defined as “attitudes and beliefs that are generally false but are widely and persistently held, and that serve to deny and justify male sexual aggression against women” (Lonsway and Fitzgerald, 1994 , pag. 134).

From a feminist perspective, Jeffreys argues that some Western beauty practices (e.g., makeup, high-heeled shoes, breast implants) should be included in the United Nations’ definition of harmful traditional/cultural practices due to the damaging effects they have on women’s health, the creation of sexual difference, and the enforcement of female deference. Gill’s article contends that it is crucial to examine how the media portrays feminism and to delve into the complexities of a cultural moment that seems to be characterised by a range of feminist ideologies (both contemporary and traditional), as well as a resurgence of anti-feminist attitudes and prevalent misogyny.

Massanari’s research centres on online misogyny and is based on a long-term participant-observation and ethnographic study of Reddit’s culture and community. The research specifically focuses on the #Gamergate and The Fappening cases. The Fappening involved the illegal distribution of nude photos of celebrities via anonymous image-board 4chan and Reddit, while #Gamergate was ostensibly about ethics in gaming journalism but became a campaign of harassment against female and minority game developers, journalists, and critics. The study highlights how Reddit’s design, algorithm, and platform politics implicitly support toxic technocultures, providing a fertile ground for anti-feminist and misogynistic movements to flourish. Online misogyny is also discussed in the papers with the highest number of local citations (i.e., citations from other documents within our bibliographic dataset): “#MasculinitySoFragile: Culture, structure, and networked misogyny” (Banet-Weiser and Miltner, 2016 ), “Back to the kitchen, cunt: Speaking the unspeakable about online misogyny” (Jane, 2014 ), and “Drinking male tears: language, the Manosphere, and networked harassment” (Marwick and Caplan, 2018 ).

Conceptual Structure of research on misogyny

To understand the conceptual structure of the research on misogyny, we initially performed an exploratory analysis of the textual content of the keywords chosen by the authors.

Figure 3 shows the most used keywords, after removing the term “misogyny”. Besides the extensive terms gender, feminism and sexism, we find keywords related to the phenomenon of violence against women, to the emerging theme of the Manosphere and to the classical theme of patriarchy and hegemonic masculinity. It is also worth noting the presence of several keywords strictly linked to the online diffusion of misogynistic content.

Most used keywords.

To deepen the analysis, a conceptual structure map (see Fig. 4 ) of the literature on misogyny was created using the Bibliometrix R package (Aria and Cuccurullo, 2017 ), which enables performing multiple correspondence analysis (MCA, Greenacre and Blasius, 2006 ) and hierarchical clustering. MCA, in particular, allows the generation of a low-dimensional Euclidean representation of the original data matrix by performing a homogeneity analysis of the “documents by keywords” indicator matrix, which is constructed by taking into account a dummy variable for each keyword. The words are plotted on a two-dimensional plane, where closer words have a more consistent distribution across the documents.

Conceptual map of research on misogyny.

The two dimensions of the maps that emerged from the MCA can be interpreted as follows. The first dimension separates keywords emphasising the problem of misogyny in general and on social media platforms (on the right) from those related to the methodological aspects of the automatic detection of misogynistic language (on the left). The second dimension separates keywords emphasising the problem of misogyny from a general point of view (on the upper) from those related to the Manosphere and Incels ( involuntary celibates ) and their presence on the Reddit platform (on the bottom). Figure 4 also displays the results obtained through a hierarchical cluster analysis carried out adopting the method of average linkage on the factorial coordinates obtained through MCA. Five clusters emerge from the conceptual structure map. The orange cluster refers to publications related to the automatic detection of misogynistic content through machine learning and deep learning techniques. The green cluster displays the connection between misogyny and hate speech and the exploitation of Natural Language Processing (NLP) methodologies to investigate these phenomena. The blue cluster refers to the intersectionality of research on misogyny. The red cluster is strictly linked to studies of the presence of misogynistic content on social media. Finally, the purple cluster is related to publications dealing with the topics of the Manosphere and the Incel phenomenon.

Research themes in misogyny literature

A topic modelling approach is exploited to investigate the textual content of title, abstract and authors’ keywords to give extra insight into multiple latent themes emerging from the literature on misogyny. To reveal the themes, research interests and trends of studies in the existing misogyny literature, we rely on the LDA model.

Topic analysis: LDA model

LDA is an unsupervised machine-learning-based algorithm allowing to discovery of latent (unobserved) “topics” in large unstructured text datasets (Blei et al., 2003 ). Previous research applied LDA to bibliometrics as an efficient tool for understanding a field’s rich underlying topical structure (see, among others, Suominen and Toivanen, 2016 , Tontodimamma et al., 2021 ). The idea behind LDA is that documents contain multiple topics, and each topic is represented as a probability distribution over terms in a fixed vocabulary, with different topics represented by different probabilities of words in the vocabulary. LDA generative process specifies a joint distribution of hidden and observed variables. The algorithm aims to estimate the posterior distribution of the hidden variables given the observed data, but exact inference is intractable, requiring approximate inference algorithms like sampling-based and variational algorithms (Blei et al., 2003 ; Steyvers and Griffiths, 2006 ). To employ LDA, the user needs to specify the number of latent topics in the corpus and two hyperparameters that control how documents and words contribute to topics. A detailed explanation of the LDA algorithm can be found in the studies by Blei ( 2012 ) and Steyvers and Griffiths ( 2006 ).

In our analysis, we use LDA to model a corpus with each document consisting of the publication title, abstract, and keywords. LDA analysis was performed through the Fitlda Matlab routine, available in the Text Analytics Toolbox (MATLAB, 2022 ).

Topic interpretation

The themes generated by LDA are hidden variables that require proper interpretation, typically done by examining the top keywords associated with each topic (Steyvers and Griffiths, 2006 ). To this end, Figs. 5 and 6 show the most important words for each topic, with importance determined by normalising the posterior word probabilities for each topic by the geometric mean of the posterior probabilities for the word across all topics. The topics are ranked based on their estimated likelihood of being observed in the entire dataset. Section 2 of the Supplementary Material contains the list of the most significant terms and their relevance measurements. The twelve selected topics address crucial areas of research on misogyny and can be summarised as follows.

Word clouds for topics 1–6.

Word clouds for topics 7–12.

Topic 1 revolves around a comprehensive discussion on the feminist perspective of the misogyny phenomenon and addresses the root causes of misogyny and gender-based discrimination. The primary focus is on patriarchal male gender privilege and its role in perpetuating misogyny. This topic covers a range of issues related to gender inequality, such as the leadership gap between men and women, women’s rights, and the intersection of misogyny with other forms of oppression.

Topic 2 focuses on how misogyny is expressed in literary works from the early and medieval periods to the modern era. Overall, this topic highlights the role of novels, prose, tales, and fiction in shaping societal attitudes and beliefs about gender and how this has influenced the treatment of women throughout history.

Along similar lines, Topic 3 centres on the study of misogyny in relation to the representation of women in films and on how it influences the portrayal of women on visual media.

Topic 4 is focused on the study of misogyny within the realm of politics and examines how misogyny can be perpetuated within political systems and movements. In particular, the inclusion of terms like “American”, “white”, and “altright” suggests that research included in this topic might focus on the ways in which misogyny is manifested in American politics, particularly within white nationalist and alternative-right movements.

Topic 5 is centred on the study of masculinity and how it relates to misogyny. In particular, the word “hegemonic” suggests that this topic may focus on how dominant forms of masculinity reinforce misogyny and gender-based discrimination.

Topic 6 pertains to the research on women’s rights, including reproductive rights, family law, and access to healthcare, particularly within legal and political systems and on how these systems can either promote or hinder gender equality.

The latent theme of Topic 7 seems to refer to a broad subject area that encompasses issues related to education, sexuality, and sexual identity. Additionally, the related terms suggest a focus on the ways in which sexuality is addressed within educational institutions, including schools and universities.

Topic 8 is a subject area that focuses on the study of digital misogyny, which refers to the ways in which sexism and gender-based discrimination are perpetuated through online and digital media platforms.

The set of words linked to Topic 9 clearly indicates studies focusing on the subject of sexual violence and harassment.

Research included in Topic 10 is related to the investigation of misogyny in the context of music and religion.

Topic 11 appears to be focused on the intersection of misogyny and racism, particularly as it relates to the misogynoir phenomenon.

Finally, Topic 12 deals with the identification and classification of online misogyny.

Topic interactions

By modelling each document as a mixture of several topics and each topic as a combination of words, the LDA technique assigns topics to documents. In our analysis, we awarded the top three document-topic probabilities to each document in this study as long as the probabilities are greater than 0.2. We developed a topic relationship network by considering the topic co-occurrence matrix. The topic network is depicted in Fig. 7 , along with node centrality measures. The nodes are coloured according to their degree, and the edges are weighted depending on co-occurrences. The stronger the link, the thicker the line. Edges with weights less than the average number of co-occurrences have been omitted. The investigation of the linkages reveals relationships between research fronts, emphasising the multidisciplinary character of research on misogyny. The highest degrees are associated with the first three topics, which encompass broader themes dealing with the feminist perspectives of patriarchal society (Topic 1) and the representation of women in literary works (Topic 2) and cinema (Topic 3). Moreover, the latter two topics show the strongest interconnection. Lower degrees are associated with more specialistic research fronts related to the presence of misogyny in music and religion, the misogynoir phenomenon, and the recent field of misogyny detection in computational sciences. In particular, the theme of automatic identification of misogynistic content (Topic 12) is only linked to the research dealing with digital misogyny (Topic 8). A high betweenness, measuring the extent to which the node is part of paths that connect an arbitrary pair of nodes in the network, is associated with Topics 5 and 6, dealing with the study of masculinity and how it relates to misogyny and to research on women rights, respectively. These findings suggest that those research areas are more effective and accessible in the network and form the densest bridges with other nodes.

Topic co-occurence network for the publications on misogyny and nodes’ centrality measures.

Topic temporal evolution

The temporal evolution of the scientific productivity for each topic can be captured through Fig. 8 . The temporal trend of most topics agrees with exponential growth. However, looking at Topic 2, related to studies of misogyny in literary works, we notice how the number of publications in the last period falls below the number expected according to the exponential law. Conversely, the number of published documents for Topics 8 and 12 shows a sudden rise starting from 2018. This trend testifies to the increasing interest in the study of online misogyny and the related techniques for automatic detection and identification. A relatively more contained rise in the size of publications is recorded for Topics 10 related to the investigation of misogyny in the context of music and religions.

Number of publications on misogyny for each topic: observed and expected distributions according to exponential growth.

The appearance and development of new fields of interest and innovative ideas in the research activity on misogyny are confirmed by the heatmaps provided in the Supplementary Material , which show the number of documents, by years, assigned to the identified topics.

Sociological research on online misogyny

To improve our comprehension of the ongoing research on the online dissemination of misogynistic content, we utilised a more specific selection query in our search of the original set of documents, which targeted terms associated with the online environment. We limited our search to articles published in journals categorised under the Social Science subject area. After analysing 277 articles, we identified 187 that were suitable for our study.

Among these documents, four articles provide a review of the literature on online misogyny from different perspectives. Moloney and Love ( 2018 ) review the way online misogyny is conceptualised in the social scientific literature within feminist media studies. The authors identify four different terms that are used to describe different types of online misogyny: online sexual harassment, gendertrolling, e-bile, and disciplinary rhetoric. They also examine the sociological perspective and introduce the concept of “virtual manhood acts” (VMAs), which is situated within the broader context of critical gender theory. VMAs are examples of technologically facilitated misogyny that occur in online social spaces: textual and visual cues are exploited to signal a masculine self, enforce traditional gender norms, oppress women, and restrict men to predefined gender roles. Bosch and Gil-Juarez ( 2021 ) conducted both a systematic review of 33 articles found in Web of Science and a traditional review of academic, institutional, and feminist-activist publications. Their findings show that the majority of aggressors in online gender-based violence are cis-hetero-patriarchal men, who are mostly known to the victims and are often partners or ex-partners. The types of violence range from sexual insults and threats to sexual and high-tech violence. Rubio Martìn and Gordo Lòpez ( 2021 ) provide an overview of the most recent academic literature within the feminist technosocial literature, specifically related to sexual and gender-based violence in digital environments. In addition to discussing the contemporary antecedents of this perspective and presenting current positions and the most representative studies on topics related to online misogyny, the authors demonstrate the potential of the feminist technosocial approach for analysing digital environments and their designs. The main conclusion drawn is that both the values of a misogynistic culture and the possibilities for its reproduction and dissemination are embedded in the design and architecture of digital platforms. The article also highlights the increasing relevance of hybrid realities that result from the synergies between the physical and digital realms, as they enable amplified discourses and actions of online misogyny. Faith ( 2022 ) investigates how gender, technology, and development are interconnected by analysing various works from different fields, including feminist technology studies, gender and development, feminist criminology, and ICT for development. The study also draws data from sources such as civil society, news reports, and international organisations, like the UN, to examine online violence. The author argues for a critical research approach to better understand the complex and opaque power dynamics that shape the digital economy and how it affects gender and development goals.

The articles on online misogyny, which were found in the Social Science category, underwent a manual annotation process to extract various pieces of information. Regarding the different methodologies and techniques used to investigate online misogyny, our findings indicate that discourse analysis and content analysis are the primary methodologies employed in social science literature. Several studies utilise in-depth interviews and surveys to examine the individuals targeted by and responsible for online misogyny. Additionally, digital ethnography, corpus linguistics, and network analysis are also employed. The most analysed social media platforms include Twitter, Reddit and Facebook. Further details on the methodological approaches and the social networks are provided in the Supplementary Material . The subsequent sections delve into details regarding target victims, misogynistic groups, and potential measures to counteract online misogyny.

Targets of online misogyny

Scholars studying online misogyny have identified various target groups that are particularly vulnerable to misogynistic content. These groups include female politicians, journalists, celebrities, influencers, musicians, gamers and developers, YouTubers, university students, and women who have been sexually assaulted. By focusing on specific target groups, research helps in achieving a more nuanced understanding of the ways in which online misogyny manifests and the specific harms that it causes.

Studies on online misogyny directed towards female politicians have concentrated on analysing the experiences of women from various countries, examining the types of misogynistic content directed towards them and the platforms on which it is disseminated. Silva-Paredes and Ibarra Herrera ( 2022 ), using a corpus-based critical discourse analysis, explore abuse received by a Chilean right-wing female politician. Phipps and Montgomery ( 2022 ) conducted an investigation into the portrayal of Nancy Pelosi as the monstrous feminine in the deeply misogynistic attack advertisements of Donald Trump’s 2020 presidential re-election campaign. In light of the prevalent misogynistic and anti-feminist depictions of Senator Hillary Clinton across all types of media, Ritchie ( 2013 ) examines how online media continues to have the power to create harmful representations of female politicians and the consequences for the political campaigns of women and for the democratic process as a whole. Focusing on Canadian politicians, Wagner ( 2022 ) discusses how online harassment is a gendered phenomenon. The study, drawing upon interviews with 101 people from diverse genders, racial/ethnic identities, sexual orientations, and partisan affiliations, shows that women are more aware of online harassment than men and how it succeeds in making women feel they are in a hostile political environment. Saluja and Thilaka ( 2021 ), analysing the Twitter discourse referring to three well-known female politicians in India, reveal similar findings, emphasising how female politicians are subjected to a different and distinct pattern of reception compared to their male counterparts. Instances of misogynistic or sexist hate speech and abusive language against female politicians in Japan are investigated in Fuchs and Schäfer ( 2021 ).

The research conducted by Chen et al. ( 2020 ) through in-depth interviews with 75 female journalists from Germany, India, Taiwan, the United Kingdom, and the USA revealed that those journalists frequently encounter online gendered harassment. The harassment, which includes sexist comments that criticise, attack, marginalise, stereotype, or threaten them based on their gender or sexuality, has led to some female journalists being subjected to misogynistic attacks and even threats of sexual violence. The study suggests that this kind of harassment limits their level of interaction with their audience without being attacked or sexually undermined.

By examining the findings of the qualitative in-depth interview of 48 female journalists, Similar findings are reported by Koirala ( 2020 ), whose study, based on the qualitative in-depth interview of 48 female journalists in Nepal, highlights how some of them tolerate harassment by being ‘strong like a man’, while many avoid social media platforms to keep free of such abusive behaviour. Along the same lines, Rego ( 2018 ) analyses Twitter conversations with Indian journalists and argues that social media platforms constitute convenient havens of harassment against assertive women.

Ghaffari ( 2022 ), analysing user-generated comments on the Instagram profile of a female American celebrity, shows how women are required to suppress their feelings and limit their authentic online presentation to maintain the outward countenance that matches the stereotypical gender roles in audiences’ state of mind. The research conducted by Döring and Mohseni ( 2019 ) supports these findings, focusing on video producers on YouTube. Their study found that female video producers are more likely to receive negative comments compared to male producers, but only if they display their sexuality or address feminist topics. However, if they conform to traditional gender role expectations, they do not experience this kind of negative feedback.

The emergence of female gamers in video game communities has led to a rise in misogynistic attacks against those who challenge the traditional hypermasculine culture of gaming. The 2014 #gamergate incident is a prime example of this, where a group of gamers opposed “Social Justice Warriors” who highlighted discrimination and exclusion in the gaming industry. Female gamers were subject to death threats, rape threats, and doxxing, where their private information was shared online (Tomkinson and Harper, 2015 ). The video gaming community has a long history of gender-based attacks on women, which serve to create a toxic environment for them when making and playing video games. According to Jenson and De Castell ( 2021 ), who approach the subject from a feminist perspective, video games have been predominantly masculine and gendered spaces. Repeated displays of aggression, referred to as “shock and awe”, perpetuate and legitimise gendered hostility. These displays also help to preserve exclusionary media practices designed to maintain the status quo.

The Manosphere

Numerous articles on online misogyny examine the Manosphere, a collection of websites and social media groups that endorse misogynistic beliefs. These networks are not uniform but consist of multiple misogynistic groups with differing perspectives and degrees of violence, which are associated with far-right, homophobic, and racist ideologies (Dickel and Evolvi, 2022 ). Despite their variations, all these groups portray feminism as innately discriminatory and threatening to men (Farci and Righetti, 2019 ). The Manosphere adheres to the beliefs of a ‘gynocentric order’ and the Red Pill ideology, a metaphor derived from the movie The Matrix, in which the protagonist’s eyes are opened to reality upon taking the “red pill”. Although these groups may have distinct beliefs, many members use the term misandry, referring to the hate against men, which has ideological and community-building functions. It reinforces a misogynistic belief system that portrays feminism as a movement that hates men and boys (Marwick and Caplan, 2018 ). The use of misandry caters to both extremist misogynistic subcultures and moderate men’s rights groups. It enables these groups to adopt the language of identity politics, positioning men as the silenced victims of reverse discrimination in all aspects of political, economic, and social life and solidifying their sense of entitlement (Farci and Righetti, 2019 ).

Men’s rights activists employ a personal action frame to construct a plausible but fictional narrative of men’s oppression (Carian, 2022 ). The movement against feminism revolves around advocating for men’s rights while denying that gendered violence exists (Garcìa-Mingo et al., 2022 ). The Manosphere engages in a crucial ideological effort that normalises, trivialises, and legitimises sexual violence against women in various forms (Garcìa-Mingo et al. 2022 ). Some of the primary themes of this ideology are: criticising and verbally abusing women, downplaying or not taking seriously accusations or reports of rape, depicting #MeToo as a feminist plot, portraying men as victims, and advocating for the restoration of patriarchal values (Dickel and Evolvi, 2022 ). Hopton and Langer ( 2022 ), analysing Twitter content to understand how the Manosphere constructs masculinity and femininity, identify three discursive strategies: co-opting discourses of oppression, naming power, and disavowal by disaggregation. These strategies are used to position men as victims, portray women as monstrous others, and re-establish gendered power hierarchies through continuous references to rape in their discourse.

One of the main groups in the Manosphere, the Incels, believes in a hetero-patriarchal racial hierarchy and justifies their lack of sexual activity through ideas rooted in biological determinism and victimisation by women and feminism (Lindsay, 2022 ). Scotto di Carlo’s analysis of Incels (Scotto di Carlo, 2023 ) reveals a conflation of apparently sarcastic metaphors, dark humour, and misogyny to describe women, as well as unique self-representations of forum participants that do not conform to typical ‘us vs them’ identity pattern (van Dijk, 1998 ): instead of highlighting the positive qualities of their in-group, the Incels describe themselves in a derogatory manner, leading to a spiral of self-pity and self-contempt that can foster a sense of brotherhood within the community. These findings stem from a content-discourse analysis of posts from threads specifically discussing women on an incel forum and from the study of nominations and predications of self-representations used in the ‘Introductions’ thread of the same forum. Halpin ( 2022 ), drawing on a qualitative analysis of comments made on a popular Incel discussion board, demonstrates how the group uses its perceived subordinate status to justify their misogyny and legitimise its degradation of women. Conducting an ethnographic content analysis of incel-identified subreddits and using femmephobia as a lens, Menzie ( 2022 ) examines how Incels employ heteropatriarchal conceptions of femininity to devalue women and to describe the social conditions that force them to remain celibate. The study focuses on the symbolic actors constructed by Incels, namely Stacy, who represents the most attractive women, Becky, who represents women of ordinary or moderate attractiveness, and Chad, who represents dominant alpha males. Five themes emerge from the analysis. First, Incels use these symbolic gendered actors to describe a sex deficit most men suffer, implying their own undesirability. Second, Incels’ femmephobia towards hyper-feminine women for not fitting heteropatriarchal requirements is evident in “Stacy”.Third, “Becky” shows a more flexible femmephobia towards women of different appearances who uphold “unrealistic standards” and date men more attractive than themselves or rely on feminism to cope with not attracting the same men as Stacy. Through “Chad”, the fourth topic examines the idea of masculinity, incorporating feelings of jealousy and recognition of victimisation under societal conditions that allow women to exploit men financially or emotionally. Finally, the analysis reveals how Incels prioritise partner display as a symbol of wealth. Along the same lines, Koller and Heritage ( 2020 ) analysed a textual corpus created from threads posted and commented on by Incels. The study examined keywords, word frequencies, and concordance lines to explore the representation of gendered social actors. The findings suggest that Incels position different groups of men in a hierarchy in which conventionally attractive men occupy the top position. Female social actors are not placed in a similar hierarchy. Furthermore, an additional appraisal analysis of the most frequently occurring male and female social actors reveals that men are judged as unable to function, while women are viewed as immoral, dishonest, and capable of causing harm to men.

Chang ( 2022 ), analysing the discourses circulating on a Reddit forum for self-proclaimed Incels, explores the perceptions created by the term “femoid”, a derogatory term generated by Incels to refer to women, constructing them as an abject “monstrous-feminine”, serving a dehumanising function and thus justifying the violence enacted upon them. Tranchese and Sugiura ( 2021 ) focus on the similarities between the language used in pornography and that of Incels, arguing that both are different manifestations of the same misogyny. Their study involves a linguistic analysis that compares a collection of posts from an Incel subreddit community with a reference collection of posts from 688 subreddits covering other subjects. From a different perspective, Byerly ( 2020 ) investigated news media language in the coverage of Incel behaviour associated with sexual aggression. The study employs qualitative textual analysis on a sample of 70 articles obtained using keyword combinations ‘incels and violence’, ‘incels and social media’, and ‘incels and sexism’ from 29 distinct news sources across 6 countries throughout the years 2018 and 2019. Research findings indicate that news stories emphasise the role of social media in helping Incels find each other and form online communities. Additionally, specific social media sites served as locations to amplify misogynistic attitudes and to boast about their murders. Speckhard et al. ( 2021 ) conducted a study that involved gathering information on Incels’ social and personal lives, adherence to incel ideology, opinions on incel-related violence, support for violent actions, and beliefs regarding the classification of Incels as violent extremists. The data was collected through a Google Forms survey that was distributed to active adult members of a prominent Incel forum. The final sample under analysis comprises 272 respondents who self-identify as Incels. The findings demonstrate that while most of them do not advocate violence and are non-violent, those who strongly hold misogynistic beliefs are more likely to endorse violent actions. Participation in Incel online forums, which validate their viewpoints, could also lead to an increase in their misogyny. O’Donnell and Shor ( 2022 ) investigated how misogynistic Incels discuss mass violence committed by their peers. Through qualitative content analysis of comments related to the 2018 Toronto van attack, in which self-declared Incel Alek Minassian drove a van into pedestrians, killing 10 and injuring 16, they found that a large majority of self-proclaimed Incels expressed support for such violence, as well as violence in general. Incels believed that mass violence was a means to achieve four main goals: gaining more attention, seeking revenge, reinforcing traditional masculinity, and bringing about political change.

MGTOW (Men Going Their Own Way), a separatist group within the Manosphere, also promotes a misogynistic agenda. Unlike Men’s Rights Activists and Incels, MGTOWs focus on individualistic and self-empowering actions, encouraging men to lead a self-sufficient life away from women. Jones et al. ( 2020 ), using content and thematic analyses of a corpus of tweets from three of the most active MGTOW users on Twitter, have linked the MGTOW ideology with toxic masculinity, showing that the online harassment it generates is deeply misogynistic and upholds heterosexual and hegemonic masculinity. The authors note that, although misogyny and violence produced by MGTOW are not extreme, the group’s appeals to rational thinking make them appear to be common sense. Wright et al. ( 2020 ) delve deeper into the structural underpinnings and nature of MGTOW debate within their discussion forums, including leadership, moderation, in-group dynamics, and the discursive form of debates, and how this contributes to the propagation of misogyny and different calls to action. The authors conducted a content analysis of comments in the official MGTOW website’s forum and a digital ethnographic approach. Their findings showed that discussions primarily revolve around women and the MGTOW community. When discussing women, users did so in an openly misogynistic way. When discussing MGTOW, conversations sought to define and rationalise it as an ideology, both for individuals and the collective. The authors also note that the communicative form was mainly communitarian, with strong group bonding, ties, and engagement.

Countering online misogyny

Strategies and tactics used by women to cope with and address gender violence online are diverse and sometimes activated simultaneously. Some of these strategies prioritise self-care and protection, while others focus on resistance and challenging such violence. From a self-care perspective, it is crucial to adopt mitigation measures that reduce harm and minimise risks, such as assessing online identities, adopting pseudonyms or collective identities, using masks, strengthening accounts, creating distance, silencing or erasing sensitive content (Bosch and Gil-Juarez, 2021 ). In the research by Chen et al. ( 2020 ), it is shown how female journalists have developed multiple strategies for coping with abuse, including modifying their social media postings, altering their reporting subjects, and utilising technological tools to prevent offensive comments on their public pages.

Merely prioritising self-care is insufficient; an active approach should be taken to resist and transform the current state of online misogyny. This involves engaging in actions that challenge the status quo and strive for meaningful change, with the ultimate goal of repoliticising the internet and social media with, for, and from a feminist perspective (Bosch and Gil-Juarez, 2021 ). From this standpoint, social media platforms can give space to the promotion of gender-based harassment but can also serve as crucial spaces for feminist education and activism and for the formation of a feminist counter-public that directly contests a misogynistic culture (Sills et al. 2016 ). In this perspective, Kurasawa et al. ( 2021 ) discuss a new form of feminist activism called evidentiary activism, which uses evidence of gender-based online violence (GBOV). Evidentiary activism engages with existing formal evidentiary cultures by advocating for legislative and regulatory reforms to address GBOV, promoting platform-based technological solutions, and challenging conventional notions of user privacy and anonymity. In addition, it involves contributing to and embracing informal evidentiary cultures, which use evidence as a tool for cultural and political mobilisation against GBOV. Strategies used include publicising instances of GBOV, highlighting the moral implications of such violence, and fostering feminist digital citizenship. As an example of online feminist activism, Kim ( 2017 ) explored the role of the 2015 hashtag #iamafeminist in promoting feminist identification and activism against misogyny in South Korea. The hashtag persisted for three months by addressing current gender issues and promoting activism both online and offline. The article by Shesterina and Fedosova ( 2021 ) examines the methods used by female bloggers to promote feminist ideas on Instagram. The authors found that while many posts are logically argued, female bloggers often use emotional manipulation and persuasion techniques to promote their ideas. The study identifies both the main topics in support of feminism, such as domestic violence and gender stereotypes, victim blaming, and the most common attitudes that female bloggers challenge in their posts (e.g., “gender roles are determined by nature”, “a woman must obey a man”, “female intelligence is worse than male”, “all women are hysterical”). The authors also describe the lexical means and rhetorical techniques commonly used in female blogs, such as metaphors, allusions, appeals, and rhetorical questions. The language used is generally colloquial, making texts easier to read, but it also includes harsh criticism and increased emotionality compared to traditional journalistic texts.

However, according to Jane ( 2016 ), taking matters into one’s own hands when faced with online harassment may have limited effectiveness and is not a sufficient solution to the problem of gendered cyber-hate. This approach shifts the responsibility from the perpetrators to the targets and the private sphere rather than addressing the broader social issue. The author suggests that a combination of feminist activism efforts, including a revised approach to collectivism, is needed to enact the necessary legislative and corporate changes to combat gendered online hate. The study by Davis and Santillana ( 2019 ) examines the potential and limitations of digital media activism in raising awareness about gender-based harassment using the case study of Las Morras, a Mexico City-based feminist media group. The study demonstrates the paradoxical role of networked digital media as an activist tool. While it rapidly circulated a critique of misogyny, it also attracted negative attention, leading to the group’s eventual demise due to doxing, trolling, and personal threats directed at its members.

Megalians, a cyberfeminist community in South Korea, utilised the technique of “mirroring” to combat online misogyny (Jeong and Lee, 2018 , Moon et al., 2022 , Yang and Lee, 2022 ). This practice involved mimicking the language of misogynistic online communities and reversing the roles of perpetrators and victims. Megalians also used parodies to subvert the humour and power dynamic that men often used to make fun of women. By appropriating and using the language of misogynists, they aimed to strip men of their ability to use misogynistic speech for their own entertainment. This approach also exposed the absurdity and ridiculousness of the misogynistic rhetoric. However, the success of mirroring is not clear-cut. In fact, while Megalians’ voices were heard in society, the strong message and crude language proved divisive and polarising (Kim, 2021 ).

An alternative strategy for addressing misogyny is to use social re-norming and appeal to the empathy of those engaging in harassing behaviour. The goal of re-norming is to challenge cultural attitudes and beliefs that tolerate or encourage violence against women and to promote new standards of behaviour that prioritise respect, equality, and safety for all individuals. One example of this approach is the experiment conducted by Whiley et al. ( 2023 ) on Twitter. Their experiment aimed to inform misogynistic offenders that their sexist language was disapproved of by the majority of people. However, this intervention did not result in a reduction in the number or frequency of sexist Tweets or users, nor did it affect the tone or emotional intensity of subsequent tweets. In contrast, research has demonstrated the efficacy of creative humour, such as that used by the IncelTears subreddit to ridicule Incels, in promoting (dis)affiliative and informative functions (Dynel, 2020 ).

Computational science research on online misogyny

In this section, we focus on documents on misogyny classified by Scopus in the “Computer Science” subject area. A total of 196 documents were found; 30 documents were excluded as they were off-topic. Two surveys were identified in the retrieved documents, which centre on the automated detection of online misogyny. In one survey, Shushkevich and Cardiff ( 2019 ) present an examination of techniques for identifying misogyny in social media through automation. Meanwhile, Sultana et al. ( 2021 ) conducted a systematic literature review of prior research to reveal different aspects of misogyny and sexist humour and to create a codebook for annotation purposes.

Automatic detection of misogyny

Manual classification of the retrieved articles reveals a wealth of valuable information regarding the automatic detection of misogyny. This includes details about the social networks that are being analysed, the primary techniques employed, and the availability of datasets.

In line with research in the social science area (see Section 4), Twitter (with 95 publications) and Reddit (with 46 publications) continue to be the most commonly used sources, even in the area of computational science. The number of studies dealing with Facebook and Instagram is very limited. Researchers frequently prioritise the study of Twitter (now rebranded X) and Reddit above other social media platforms due to their historically liberal provision of Application Programming Interface (API) access. Furthermore, Reddit, which has been described as ’a community of communities’ (Massanari, 2017 , p. 331), has a diverse array of subreddits that cater to different interests, some of which foster misogynistic beliefs. However, the new pricing plans for using the Twitter API, introduced in March 2023, are expected to significantly affect research. A survey conducted by the Coalition for Independent Technology Research Footnote 2 outlines the potential consequences of discontinuing free and affordable API access. These drawbacks include the disruption of research on the dissemination of harmful content. A similar survey on the impact of Reddit’s recent API changes Footnote 3 emphasises how researchers are concerned about interruptions in their research resulting from API modifications. It is worth noting that only one study (Semenzin and Bainotti, 2020 ) reports the results of research on Telegram, which, in fact, has become a widely used platform for the dissemination of abusive and misogynistic content due to its high degree of anonymity and limited content-moderation policies (Guhl and Davey, 2020 ).

The automatic detection of misogyny typically utilises various techniques, with pre-trained deep-learning models and multimodal models being the most commonly employed. Other techniques include machine learning algorithms such as SVM, Naïve Bayes, or Random Forest. Additionally, some documents rely on convolutional neural network models. More details on the published documents employing the different techniques are provided in the Supplementary Material .

Four articles employ the use of lexicons for automatic detection of misogyny. Attanasio and Pastor ( 2020 ) propose misogyny lexicons for automatic misogyny identification in order to improve sentence embedding similarity. Hurtlex (Bassignana et al. 2018 ), which is a lexicon of offensive, aggressive, and hateful words in more than 50 languages, is exploited for misogyny identification in the studies by Chiril et al. ( 2022 ) and Pamungkas et al. ( 2018 ). Kwarteng et al. ( 2022 ) created a specific lexicon around misogynoir.

Taxonomies and guidelines

When releasing annotated datasets, a crucial aspect is to clearly outline the guidelines for categorising misogynistic language. Four articles in the retrieved documents address this issue (Anzovino et al., 2018 , Guest et al., 2021 , Sultana et al., 2021 , Zeinert et al., 2021 ).

Sultana et al. ( 2021 ) proposed eleven categories to classify misogynistic remarks: Discredit (slurring over women with no other larger intention), Stereotyping (description of women’s physical appeal and/or comparisons to narrow standards), Sexual harassment (to physically assert power over women), Threats of violence (intent to physically assert power over women or to intimidate and silence women through threats of violence), Dominance (to preserve male control, protect male interest and exclude women from the conversation), Derailing (to justify abuse, reject male responsibility, and attempt to disrupt the conversation in order to refocus it), Victim blaming (blaming the victims for the problems they are facing), Mixed bias (gender bias might be mixed with other kinds of biases like religious or racial), Sexual objectification (evoke sexual imagery), and Damning (contains prayers to hurt women). Regarding the expression of misogyny using humour, this research proposes eight categories of jokes: Devaluation of personal characteristics, Women’s place in the private sphere, Violence against women, Feminist backlash, Sexual objectification, Excluding and/or objectifying humour, Transphobic Jokes and Cruel or Humiliation. All the categories proposed in Anzovino et al. ( 2018 ) are included in Sultana et al. ( 2021 ). The same occurs with categories proposed by Zeinert et al. ( 2021 ), except for the interesting concept of neosexism. Neosexism is a concept defined in Francine Tougas et al. ( 1999 ), and presents as the belief that women have already achieved equality and that discrimination of women does not exist. Neosexism was the most common form of misogyny present in the dataset of Zeinert et al. ( 2021 ). Guest et al. ( 2021 ) define four categories of misogynistic content: misogynistic pejoratives, descriptions of misogynistic treatment, acts of misogynistic derogation and gendered personal attacks against women.

Evaluation campaigns

A number of the documents on misogyny that fall within the Computer Science subject area were produced in connection with various evaluation campaigns. These campaigns include EVALITA (Evaluation of NLP and Speech Tools for Italian), IberLEF (Iberian Languages Evaluation Forum), SemEval (International Workshop on Semantic Evaluation), and FIRE (Forum for Information Retrieval Evaluation). The EVALITA campaign includes the Automatic Misogyny Identification (AMI) task (Fersini et al. 2018 ). The IberLEF annual campaign features the EXIST task, which is sEXism Identification in Social neTworks (Rodrìguez-Sanchez et al. 2021 ). SemEval has a task called MAMI, which is Multimedia Automatic Misogyny Identification (Fersini et al., 2022 ). Lastly, FIRE includes the Arabic Misogyny Identification (ArMI) task (Mulki and Ghanem, 2022 ).

Thanks to these evaluation campaigns, datasets for automatic misogyny detection in multiple languages are now available. Specifically, the AMI task made available two datasets, in English and Italian, downloaded from Twitter. The EXIST task provided datasets of tweets in both English and Spanish. The dataset released for the MAMI challenge comprises memes that were downloaded from popular social media platforms such as Twitter and Reddit, as well as from websites dedicated to meme creation and sharing. Lastly, the ArMI task provided a dataset of tweets written in Modern Standard Arabic (MSA) and various Arabic dialects.

The bibliometric analysis reveals that research on misogyny has witnessed exponential growth from 2010 to 2022. This growth can be attributed to various areas of research, but one prominent factor contributing to this trend is the increased attention given to the online dissemination of hate towards women. Several findings support this initial conclusion.

Firstly, the analysis indicates that the most productive authors in the field of misogyny research come from the area of computer science. This suggests that experts in this field have been actively investigating and publishing on the topic, further driving the growth of research in this area.

Moreover, examining the topics covered in the analysed documents provides additional evidence for the influence of online misogyny. Topic 8, which is related to digital misogyny, and Topic 12, which focuses on the automatic identification of misogyny in social media, have experienced significantly higher growth compared to the broader field of misogyny research (as depicted in Fig. 8 ). This finding indicates that the study of misogyny in online platforms and the development of methods to detect misogyny in social media have gained considerable attention within the research community.

The major role that online misogyny plays in the development of the area supports the idea that the research seeks to delineate the contours of a new face of misogyny, the latest manifestation of hate towards women which is expressed more crudely and more openly on social networks because they facilitate anonymity and a greater distance from the victims.

Another conclusion drawn from the analysis of the conceptual structure of misogyny research (Fig. 4) and the interactions between topics (Fig. 7 ) is that the research focused on the automatic detection of misogyny in online platforms (Topic 12) exhibits weak connections with other conceptual areas that address different aspects of the phenomenon. This area of research only demonstrates some conceptual relation to the broader study of online misogyny (Topic 8). This presents a significant challenge, considering that qualitative analysis of sociological research emphasises the growing relevance of hybrid realities resulting from the synergies between the physical and digital realms, not just in violence against women but also in specific domains such as politics. Moreover, the lack of relationship between Topic 12, which focuses on the automatic detection of misogyny, and Topic 9, which explores violence against women and the concept of Manosphere (primarily a digital phenomenon), is particularly noteworthy. This suggests that research in the computational science domain may not be adequately addressing the most extreme manifestations of online misogyny. Furthermore, it also indicates that the tools offered by computational linguistics are underutilised in social science-led research.

In general, the absence of stronger connections between certain topics that attract the attention of various disciplines could be seen as a sign of the practical challenges encountered in interdisciplinary research. For instance, Topic 6, which focuses on the study of women’s rights within legal and political systems, exhibits very weak relationships with Topics 8 and 12, despite qualitative sociological research emphasising the need to consider the new dynamics emerging in virtual spaces. Another illustration can be found in the qualitative review of computational science literature. It becomes apparent that this research area relies on the definition of taxonomies that would benefit from clarification through collaboration with social science research. For instance, the inclusion of stereotypes against women as part of the types of misogyny raises the question of whether the concept of misogyny should be reserved for the most extreme forms of hatred or should encompass the wide range of sexist attitudes and gender symbolic constructions derived from a patriarchal culture.

The main conclusion drawn from this work is that research across different disciplines is addressing a new facet of misogyny, a revitalised version of outdated beliefs about women’s inferiority that circulate in novel forms within the online realm. Understanding the characteristics and functions of this new expression of misogyny poses a challenge that necessitates an interdisciplinary approach, leveraging the strengths of different areas of knowledge to effectively address it.

The above-mentioned lack of collaboration between different areas prevents the establishment of connections that would enrich the analysis of the way misogyny is disseminated today in both the virtual and real world. For example, social science knowledge in combination with computational discourse analysis or NLP technologies could be used to study the connections and similarities between agents disseminating misogyny online and mainstream social actors such as political parties or religious organisations. In the same way, the similarity between misogynist discourses and those of left-leaning feminists in open battle against other fractions of the feminist movement could also be monitored and would allow for a more complex view of the phenomenon. For both approaches, it is necessary that social science knowledge strongly rooted in the study of social relations be combined with the new methodologies that computer science offers for the analysis of discourse produced naturally in digital or real communicative exchanges, such as in parliaments, rallies or interviews.

Data availability

Data sharing is not applicable to this research, as no data were generated. The analysed data were retrieved from the commercial Web of Science (WOS) and Scopus databases, following the search procedure detailed in the Supplementary Material .

https://www.theguardian.com/technology/2015/jan/11/gamergate-a-brief-history-of-a-computer-age-war , https://time.com/3510381/gamergate-faq/

https://independenttechresearch.org/letter-twitters-new-api-plans-will-devastate-public-interest-research/

https://independenttechresearch.org/reddit-survey-results/

Alghamdi R, Alfalqi K (2015) A survey of topic modeling in text mining. Int J Adv Comput Sci Appl 6(1):147–153

Google Scholar  

Allen A (2021) Feminist perspectives on power. In: Zalta E (ed) The Stanford encyclopedia of philosophy. Metaphysics Research Lab, Stanford University

Anzovino M, Fersini E, Rosso P (2018) Automatic identification and classification of misogynistic language on Twitter. In: Silberztein M, Atigui F, Kornyshova E et al (eds) Natural language processing and information systems (NLDB) 2018. Lecture notes in computer science, vol 10859. Springer, Cham, pp. 57–64

Aria M, Cuccurullo C (2017) bibliometrix: an R-tool for comprehensive science mapping analysis. J Informetr 11(4):959–975

Article   Google Scholar  

Attanasio G, Pastor E (2020) PoliTeam @ AMI: improving sentence embedding similarity with misogyny lexicons for automatic misogyny identification in Italian tweets. In: Proceedings of the 7th evaluation campaign of natural language processing and speech tools for Italian (EVALITA 2020) (eds Basile V, Croce D, Maro M, Passaro LC) CEUR workshop proceedings, vol 2765, Accademia University Press, Torino

Banet-Weiser S, Miltner KM (2016) #MasculinitySoFragile: culture, structure, and networked misogyny. Fem Media Stud 16(1):171–174

Bassignana E, Basile V, Patti V (2018) Hurtlex: a multilingual lexicon of words to hurt. In: Proceedings of the fifth Italian Conference on Computational Linguistics (CLiC-it 2018), Torino, December 10–12, 2018, (eds Cabrio E, Mazzei A, Tamburini F) CEUR Workshop Proceedings, vol 2253, Accademia University Press, Torino, Italy

Bates L (2021) Men who hate women: from incels to pickup artists: the truth about extreme misogyny and how it affects us all. Sourcebooks, Naperville, IL

Blei DM (2012) Probabilistic topic models. Commun ACM 55(4):77–84

Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3(1):993–1022

Bosch NV, Gil-Juarez A (2021) Un acercamiento situado a las violencias machistas online y a las formas de contrarrestarlas. Rev Estud Fem 29(3). https://doi.org/10.1590/1806-9584-2021V29N374588

Byerly CM(2020) Incels online reframing sexual violence Commun Rev 23(4):290–308

Carian EK(2022) "We’re all in this together”: leveraging a personal action frame in two men’s rights forums Mobilization 27(1):47–68

Chang W(2022) The monstrous-feminine in the incel imagination: Investigating the representation of women as “femoids” on /r/Braincels Fem Media Stud 22(2):254–270

Article   MathSciNet   Google Scholar  

Chen GM, Pain P, Chen VY(2020) ‘You really have to have a thick skin’: a cross-cultural perspective on how online harassment influences female journalists Journalism 21(7):877–895

Chiril P, Pamungkas E, Benamara F (2022) Emotionally informed hate speech detection: a multi-target perspective. Cogn Comput 14(1):322–352

Citron DK (2014) Hate crimes in cyberspace. Harvard University, Cambridge, MA

Book   Google Scholar  

Code L (2000) Encyclopedia of feminist theories. Routledge, London

Davis S, Santillana M(2019) From the streets to the screen to nowhere: Las morras and the fragility of networked digital activism Westminst Pap Commun Cult 14(1):18–32

Dickel V, Evolvi G (2022) “Victims of feminism”: exploring networked misogyny and #MeToo in the manosphere. Fem Media Stud https://doi.org/10.1080/14680777.2022.2029925

Döring N, Mohseni MR(2019) Male dominance and sexism on YouTube: results of three content analyses Fem Media Stud 19(4):512–524

Dynel M(2020) Vigilante disparaging humour at r/IncelTears: humour as critique of incel ideology Language Commun 74:1–14

Faith B(2022) Tackling online gender-based violence; understanding gender, development, and the power relations of digital spaces Gend Technol Dev 26(3):325–340

Farci M, Righetti N(2019) Italian men’s rights activism and online backlash against feminism Rass Ital Sociol 60(4):765–781

Fersini E, Nozza D, Rosso P (2018) Overview of the EVALITa 2018 task on automatic misogyny identification (AMI). In: CEUR workshop proceedings (eds Caselli T, Novielli N, Patti V, Rosso P) vol 2263 (CEUR-WS), Accademia University Press, Torino

Fersini E, Gasparini F, Rizzi G et al (2022) SemEval-2022 Task 5: multimedia automatic misogyny identification. In: Proceedings of the 16th international workshop on Semantic Evaluation (SemEval-2022) (eds Emerson G, Schluter N, Stanovsky G. et al) Association for Computational Linguistics, pp. 533–549

Francine Tougas F, Brown R, Beaton AM (1999) Neosexism among women: the role of personally experienced social mobility attempts. Personal Soc Psychol Bull 25(12):1487–1497

Fuchs T, Schäfer F(2021) Normalizing misogyny: hate speech and verbal abuse of female politicians on Japanese Twitter Jpn Forum 33(4):553–579

Garcìa-Mingo E, Fernàndez SD, Tomàs-Forte S (2022) (Re)configurando el imaginario sobre la violencia sexual desde el antifeminismo: el trabajo ideològico de la manosfera española. Polit Sociedad 59(1). https://doi.org/10.5209/poso.80369

Ghaffari S(2022) Discourses of celebrities on Instagram: digital femininity, self-representation and hate speech Crit Discourse Stud 19(2):161–178

Gill R (2016) Post-postfeminism?: new feminist visibilities in postfeminist times. Fem Media Stud 16(4):610–630

Ging D, Siapera E (2018) Special issue on online misogyny. Fem Media Stud 18(4):515–524

Glick P, Fiske ST (1997) Hostile and benevolent sexism: measuring ambivalent sexist attitudes toward women. Psychol Women Q 21(1):119–135

Greenacre M, Blasius J (2006) Multiple correspondence analysis and related methods. Chapman and Hall/CRC, New York

Guest E, Vidgen B, Mittos A et al An expert annotated dataset for the detection of online misogyny. In: Proceedings of the 16th conference of the European Chapter of the Association for Computational Linguistics: main volume. Association for Computational Linguistics, pp. 1336–1350 (2021)

Guhl J, Davey J (2020) A safe space to hate: white supremacist mobilisation on telegram. Technical report. ISD Global

Halpin M(2022) Weaponized subordination: how incels discredit themselves to degrade women Gend Soc 36(6):813–837

Hopton K, Langer S(2022) “Kick the XX out of your life”: an analysis of the manosphere’s discursive constructions of gender on Twitter Fem Psychol 32(1):3–22

Hu Y, Boyd-Graber J, Satinoff B (2014) Interactive topic modeling. Mach Learn 95(3):423–469

Jane EA(2014) Back to the kitchen, cunt: speaking the unspeakable about online misogyny Continuum 28(4):558–570

Jane EA (2015) Flaming? What flaming? The pitfalls and potentials of researching online hostility. Ethics Inf Technol 17(1):65–87

Jane EA(2016) Online misogyny and feminist digilantism Continuum 30(3):284–297

Jane EA (2017) Misogyny online: a short (and Brutish) history. Sage, London

Jeffreys S (2005) Beauty and misogyny: harmful cultural practices in the West. Routledge, New York, NY, USA

Jenson J, De Castell S(2021) Patriarchy in play: video games as gendered media ecologies Explor Media Ecol 20(2):195–212

Jeong E, Lee J(2018) We take the red pill, we confront the DickTrix: online feminist activism and the augmentation of gendered realities in South Korea Fem Media Stud 18(4):705–717

Jones C, Trott V, Wright S(2020) Sluts and soyboys: MGTOW and the production of misogynistic online harassment New Media Soc 22(10):1903–1921

Jukes A (1993) Why men hate women. Free Association Books Ltd, London

Kim J(2017) #iamafeminist as the “mother tag”: feminist identification and activism against misogyny on Twitter in South Korea Fem Media Stud 17(5):804–820

Kim Y(2021) Mirroring misogyny in Hell Choson: Megalia, womad, and Korea’s feminism in the age of digital populism Eur J Korean Stud 20(2):101–133

Koirala S(2020) Female journalists’ experience of online harassment: a case study of Nepal Media Commun 8(1):47–56

Koller V, Heritage F(2020) Incels, in-groups, and ideologies. The representation of gendered social actors in a sexuality-based online community J Language Sex 9(2):152–178

Kurasawa F, Rondinelli E, Kilicaslan G(2021) Evidentiary activism in the digital age: on the rise of feminist struggles against gender-based online violence Inf Commun Soc 24(14):2174–2194

Kwarteng J, Perfumi S, Farrell T et al (2022) Misogynoir: challenges in detecting intersectional hate. Soc Netw Anal Min 12(1). https://doi.org/10.1007/s13278-022-00993-7

Lindsay A(2022) Swallowing the Black Pill: Involuntary Celibates’ (Incels) anti-feminism within Digital Society Int J Crime Justice Soc Democr 11(1):210–224

Lonsway KA, Fitzgerald LF (1994) Rape myths: in review. Psychol Women Q 18:133–164

Lonsway KA, Fitzgerald LF (1995) Attitudinal antecedents of rape myth acceptance: a theoretical and empirical reexamination. J Personal Soc Psychol 68(4):704–711

Manne K (2017) Down girl: the logic of misogyny. Oxford Academic, New York

Mantilla K (2013) Gendertrolling: misogyny adapts to new media. Fem Stud 39(2):563–571

Marwick AE, Caplan R(2018) Drinking male tears: language, the manosphere, and networked harassment Fem Media Stud 18(4):543–559

Massanari A (2017) #Gamergate and The Fappening: how Reddit’s algorithm, governance, and culture support toxic technocultures. New Media Soc 19(3):329–346

Massanari AL (2020) Gamergate. John Wiley & Sons, Ltd, pp. 1–5

MATLAB (2022) MATLAB version R2022b. The MathWorks Inc., Natick, MA

Menzie L(2022) Stacys, Beckys, and Chads: the construction of femininity and hegemonic masculinity within incel rhetoric Psychol Sex 13(1):69–85

Millet K (1970) Sexual Politics. Doubleday, New York, NY

Moloney ME, Love TP (2018) Assessing online misogyny: perspectives from sociology and feminist media studies. Sociol Compass 12(5):e12,577

Moon YE, Kim HH, Park D (2022) “Can I become a true feminist?”: an interpretive analysis on the mirroring experience of young Korean women. Fem Media Stud https://doi.org/10.1080/14680777.2022.2042830

Mulki H, Ghanem B (2022) Working notes of the workshop Arabic Misogyny Identification (ArMI-2021). In: Proceedings of the 13th annual meeting of the Forum for Information Retrieval Evaluation. Association for Computing Machinery, New York, NY, USA, pp. 7–8

O’Donnell C, Shor E(2022) "This is a political movement, friend”: why “incels” support violence Br J Sociol 73(2):336–351

Article   PubMed   Google Scholar  

Ostini J, Hopkins S (2015) Online harassment is a form of violence. The Conversation 8: 1–4

Pamungkas EW, Cignarella AT, Basile V et al (2018) Automatic identification of misogyny in English and Italian Tweets at EVALITA 2018 with a multilingual hate lexicon. In: Proceedings of the sixth evaluation campaign of natural language processing and speech tools for Italian. Final workshop (EVALITA 2018), Turin co-located with the fifth Italian conference on Computational Linguistics (CLiC-it 2018), Turin (eds Caselli T, Novielli N, Patti V, Rosso P) December 12–13, 2018, CEUR Workshop Proceedings, vol 2263, Accademia University Press, Torino, Italy

Phipps EB, Montgomery F(2022) "Only YOU Can Prevent This Nightmare, America”: Nancy Pelosi as the monstrous-feminine in Donald Trump’s YouTube attacks Women’s Stud Commun 45(3):316–337

Price DJ (1963) Little science, big science. Columbia University Press, New York

Rego R(2018) Changing forms and platforms of misogyny: sexual harassment of women journalists on twitter Media Watch 9(3):472–485

Ritchie J(2013) Creating a monster: Online media constructions of Hillary Clinton during the Democratic Primary Campaign, 2007-8 Fem Media Stud 13(1):102–119

Rodrìguez-Sanchez FJ, Carrillo-de Albornoz J, Plaza L (2021) Overview of EXIST 2021: sEXism Identification in Social neTworks. Proces Leng Nat 67:195–207

Rubio Martìn MJ, Gordo Lòpez A (2021) La perspectiva tecnosocial feminista como antídoto para la misoginia online. Rev Esp Sociol 30(3). https://doi.org/10.22325/fes/res.2021.64

Saluja N, Thilaka DN(2021) Women leaders and digital communication: gender stereotyping of female politicians on Twitter J Content Community Commun 13(7):227–241

Scotto di Carlo G(2023) An analysis of self-other representations in the incelosphere: between online misogyny and self-contempt Discourse Soc 34(1):3–21

Semenzin S, Bainotti L (2020) The use of telegram for non-consensual dissemination of intimate images: gendered affordances and the construction of masculinities. Soc Media + Soc 6(4):2056305120984,453

Shesterina AM, Fedosova OA (2021) Promotion of feminist ideas in instagram texts. Vestnik Mosk Univ Seriya 10 Zhurnalistika 2021(4):114–134

Shushkevich E, Cardiff J (2019) Automatic misogyny detection in social media: a survey. Comput Sist 23(4):1159–1164

Sills S, Pickens C, Beach K(2016) Rape culture and social media: young critics and a feminist counterpublic Fem Media Stud 16(6):935–951

Silva-Paredes D, Ibarra Herrera D (2022) Resisting anti-democratic values with misogynistic abuse against a Chilean right-wing politician on Twitter: the #CamilaPeluche incident Discourse Commun 16(4):426–444

Speckhard A, Ellenberg M, Morton J, Ash A (2021) Involuntary Celibates’ experiences of and grievance over sexual exclusion and the potential threat of violence among those active in an Online Incel Forum. J Strateg Secur 14(2):89–121

Steyvers M, Griffiths T (2006) Probabilistic topic models. In: Landauer T, McNamara D, Dennis S et al (eds) Latent semantic analysis: a road to meaning. Lawrence Erlbaum

Sultana S, Sarker J, Bosu A (2021) A Rubric to identify misogynistic and sexist texts from software developer communications. In: Proceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). Association for Computing Machinery, New York, NY, USA

Suominen A, Toivanen H (2016) Map of science with topic modeling: comparison of unsupervised learning and human-assigned subject classification. J Assoc Inf Sci Technol 67(10). https://doi.org/10.1002/asi.23596

Tomkinson S, Harper T(2015) The position of women in video game culture: Perez and Day’s Twitter Incident Continuum 29(4):617–634

Tontodimamma A, Nissi E, Sarra A (2021) Thirty years of research into hate speech: topics of interest and their evolution. Scientometrics 126(157–179):69–81

Tranchese A, Sugiura L(2021) "I don’t hate all women, just those stuck-up bitches”: how incels and mainstream pornography speak the same extreme language of misogyny Violence Against Women 27(14):2709–2734

Article   PubMed   PubMed Central   Google Scholar  

van Dijk TA (1998) Editorial: discourse and ideology. Discourse Soc 9(3):307–308

VandenBos GR (ed) (2015) APA dictionary of psychology, 2nd edn. American Psychological Association

Wagner A(2022) Tolerating the trolls? Gendered perceptions of online harassment of politicians in Canada Fem Media Stud 22(1):32–47

Whiley LA, Walasek L, Juanchich M(2023) Contributions to reducing online gender harassment: social re-norming and appealing to empathy as tried-and-failed techniques Fem Psychol 33(1):83–104

Wright S, Trott V, Jones C(2020) ‘The pussy ain’t worth it, bro’: assessing the discourse and structure of MGTOW Inf Commun Soc 23(6):908–925

Yang S, Lee K(2022) The intertextuality and interdiscursivity of “mirroring” in South Korean cyberfeminist posts Discourse Soc 33(5):671–689

Zeinert P, Inie N, Derczynski L (2021) Annotating online misogyny. In: Proceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on natural language processing (vol 1: Long Papers). Association for Computational Linguistics, pp. 3181–3197

Download references

Acknowledgements

This research was undertaken as part of the ICOMIC (Identifying and Counteracting Online Misogyny in Cyberspace) Project funded by EU Next Generation, MUR-Fondo Promozione e Sviluppo-DM 737/2021

Author information

Authors and affiliations.

Department of Legal and Social Sciences, G. d’Annunzio University of Chieti-Pescara, Chieti-Pescara, Italy

Lara Fontanella & Alice Tontodimamma

PRHLT, Universitat Politècnica de València, Valencia, Spain

Berta Chulvi

Social Psychology Department, Universitat de València, Valencia, Spain

Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti-Pescara, Chieti-Pescara, Italy

Elisa Ignazzi

Department of Philosophical, Pedagogical and Economic-Quantitative Sciences, G. d’Annunzio University of Chieti-Pescara, Chieti-Pescara, Italy

Annalina Sarra

You can also search for this author in PubMed   Google Scholar

Contributions

LF and BC contributed to the study conception and design, with LF leading the study supervision. LF, BC, and AS contributed to the writing of the manuscript. LF, EI, and AT developed the dataset and conducted the statistical analysis. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lara Fontanella .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethics approval

Ethical approval was not required as the study did not involve human participants.

Informed consent

Informed consent was not required as the study did not involve human participants.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary material, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Fontanella, L., Chulvi, B., Ignazzi, E. et al. How do we study misogyny in the digital age? A systematic literature review using a computational linguistic approach. Humanit Soc Sci Commun 11 , 478 (2024). https://doi.org/10.1057/s41599-024-02978-7

Download citation

Received : 14 July 2023

Accepted : 18 March 2024

Published : 02 April 2024

DOI : https://doi.org/10.1057/s41599-024-02978-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Get 7 Days Free

Rallybio Announces FNAIT Systematic Literature Review to be Presented at the AMCP 2024 Annual Meeting

Rallybio Corporation (Nasdaq: RLYB), a clinical-stage biotechnology company committed to identifying and accelerating the development of life-transforming therapies for patients with severe and rare diseases, today announced that data from a systematic literature review assessing the frequency of fetal and neonatal alloimmune thrombocytopenia (FNAIT) risk among pregnant mothers will be presented at the Academy of Managed Care Pharmacy (AMCP) 2024 Annual Meeting in New Orleans from April 15 th to 18 th . Rallybio is developing RLYB212, a novel human monoclonal anti-HPA-1a antibody designed to prevent alloimmunization in at-risk pregnant mothers, thereby eliminating the risk of FNAIT and its potentially devastating consequences in fetuses and newborns.

Details of the poster presentation are as follows:

• Title: Fetal and Neonatal Alloimmune Thrombocytopenia: A Systematic Literature Review and Meta-analysis of Adverse Pregnancy-Related Outcomes to Support the Development of a Novel Prophylactic Therapeutic
• Presenting Author: Andrea V. Margulis, RTI Health Solutions
• Poster Number: P1
• Poster Session Date and Time: April 17, 2024, 1:00 - 2:30 p.m. CDT (2:00 - 3:30 p.m. EDT)

Additional information about the AMCP 2024 Annual Meeting is available at: https://amcpannual.org/

About FNAIT

Fetal and Neonatal Alloimmune Thrombocytopenia (FNAIT) is a potentially life-threatening rare disease that can cause uncontrolled bleeding in fetuses and newborns. FNAIT can arise during pregnancy due to an immune incompatibility between an expectant mother and her fetus in a specific platelet antigen called human platelet antigen 1, or HPA-1.

There are two predominant forms of HPA-1, known as HPA-1a and HPA-1b, which are expressed on the surface of platelets. Individuals who are homozygous for HPA-1b, meaning that they have two copies of the HPA-1b allele and no copies of the HPA-1a allele, are also known as HPA-1a negative. Upon exposure to the HPA-1a antigen, these individuals can develop antibodies to that antigen in a process known as alloimmunization. In HPA-1a negative expectant mothers bearing a HPA-1a positive fetus, alloimmunization can occur upon mixing of fetal blood with maternal blood. When alloimmunization occurs in an expectant mother, the anti-HPA-1a antibodies that develop in the mother can cross the placenta and destroy platelets in the fetus. The destruction of platelets in the fetus can result in severely low platelet counts, or thrombocytopenia, and potentially lead to devastating consequences including miscarriage, stillbirth, death of the newborn, or severe lifelong neurological disability in those babies who survive. There is currently no approved therapy for the prevention or prenatal treatment of FNAIT.

About Rallybio

Rallybio (Nasdaq: RLYB) is a clinical-stage biotechnology company with a mission to develop and commercialize life-transforming therapies for patients with severe and rare diseases. Rallybio has built a broad pipeline of promising product candidates aimed at addressing diseases with unmet medical need in areas of maternal fetal health, complement dysregulation, hematology, and metabolic disorders. The Company has two clinical stage programs: RLYB212, an anti-HPA-1a antibody for the prevention of fetal and neonatal alloimmune thrombocytopenia (FNAIT) and RLYB116, an inhibitor of complement component 5 (C5), with the potential to treat several diseases of complement dysregulation, as well as additional programs in preclinical development. Rallybio is headquartered in New Haven, Connecticut. For more information, please visit www.rallybio.com and follow us on LinkedIn and Twitter .

Forward-Looking Statements

This press release contains forward-looking statements that are based on our management’s beliefs and assumptions and on currently available information. All statements, other than statements of historical facts contained in this press release are forward-looking statements. In some cases, forward-looking statements can be identified by terms such as “may,” “will,” “should,” “expect,” “plan,” “anticipate,” “could,” “intend,” “target,” “project,” “contemplate,” “believe,” “estimate,” “predict,” “potential” or “continue” or the negative of these terms or other similar expressions, although not all forward-looking statements contain these words. Forward-looking statements in this press release include, but are not limited to, statements concerning whether a prophylactic treatment for FNAIT can be successfully developed, and whether RLYB212 will prevent alloimmunization in at-risk pregnant mothers and eliminate the risk of FNAIT. The forward-looking statements in this press release are only predictions and are based largely on management’s current expectations and projections about future events and financial trends that management believes may affect Rallybio’s business, financial condition, and results of operations. These forward-looking statements speak only as of the date of this press release and are subject to a number of known and unknown risks, uncertainties and assumptions, including, but not limited to, our ability to successfully initiate and conduct our planned clinical studies, and complete such clinical studies and obtain results on our expected timelines, or at all, whether our cash resources will be sufficient to fund our operating expenses and capital expenditure requirements and whether we will be successful raising additional capital, our ability to enter into strategic partnerships or other arrangements, competition from other biotechnology and pharmaceutical companies, and those risks and uncertainties described in Rallybio’s filings with the U.S. Securities and Exchange Commission (SEC), including Rallybio’s Annual Report on Form 10-K for the period ended December 31, 2023, and subsequent filings with the SEC. The events and circumstances reflected in our forward-looking statements may not be achieved or occur and actual future results, levels of activity, performance and events and circumstances could differ materially from those projected in the forward-looking statements. Except as required by applicable law, we are not obligated to publicly update or revise any forward-looking statements contained in this press release, whether as a result of any new information, future events, changed circumstances or otherwise.

Investor Ami Bavishi Head of Investor Relations and Corporate Communications (475) 47-RALLY (Ext. 282) [email protected] Hannah Deresiewicz Stern Investor Relations, Inc. (212) 362-1200 [email protected] Media Victoria Reynolds Mission North (760) 579-2134 [email protected]

View source version on businesswire.com: https://www.businesswire.com/news/home/20240329464694/en/

Market Updates

Q1 2024 in review and q2 market outlook, why we expect the job market’s slowdown to renew in 2024, 5 undervalued stocks to buy during q2 2024, 13 charts on the q1 stock rally that just wouldn’t quit, when will the fed start cutting interest rates, 10 top-performing dividend stocks for the quarter, markets brief: why q1 was a triumph for optimists, q2 stock market outlook: contrarian plays look increasingly attractive, stock picks, top 15 wealth-creating stocks of the past decade, the best industrials stocks to buy, energy stocks: opec’s failure to defend oil prices has created a supply glut, hurting performance, best- and worst-performing stocks for the month, technology stocks: strength continues in q1, resulting in few obvious buying opportunities, communication services stocks: advertising continues to rule, ge aerospace stock: stand-alone jet engine maker slightly undervalued, tesla: first-quarter deliveries show surprising decline, sponsor center.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Med Libr Assoc
• v.106(4); 2018 Oct

A systematic approach to searching: an efficient and complete method to develop literature searches

Associated data.

Creating search strategies for systematic reviews, finding the best balance between sensitivity and specificity, and translating search strategies between databases is challenging. Several methods describe standards for systematic search strategies, but a consistent approach for creating an exhaustive search strategy has not yet been fully described in enough detail to be fully replicable. The authors have established a method that describes step by step the process of developing a systematic search strategy as needed in the systematic review. This method describes how single-line search strategies can be prepared in a text document by typing search syntax (such as field codes, parentheses, and Boolean operators) before copying and pasting search terms (keywords and free-text synonyms) that are found in the thesaurus. To help ensure term completeness, we developed a novel optimization technique that is mainly based on comparing the results retrieved by thesaurus terms with those retrieved by the free-text search words to identify potentially relevant candidate search terms. Macros in Microsoft Word have been developed to convert syntaxes between databases and interfaces almost automatically. This method helps information specialists in developing librarian-mediated searches for systematic reviews as well as medical and health care practitioners who are searching for evidence to answer clinical questions. The described method can be used to create complex and comprehensive search strategies for different databases and interfaces, such as those that are needed when searching for relevant references for systematic reviews, and will assist both information specialists and practitioners when they are searching the biomedical literature.

INTRODUCTION

Librarians and information specialists are often involved in the process of preparing and completing systematic reviews (SRs), where one of their main tasks is to identify relevant references to include in the review [ 1 ]. Although several recommendations for the process of searching have been published [ 2 – 6 ], none describe the development of a systematic search strategy from start to finish.

Traditional methods of SR search strategy development and execution are highly time consuming, reportedly requiring up to 100 hours or more [ 7 , 8 ]. The authors wanted to develop systematic and exhaustive search strategies more efficiently, while preserving the high sensitivity that SR search strategies necessitate. In this article, we describe the method developed at Erasmus University Medical Center (MC) and demonstrate its use through an example search. The efficiency of the search method and outcome of 73 searches that have resulted in published reviews are described in a separate article [ 9 ].

As we aimed to describe the creation of systematic searches in full detail, the method starts at a basic level with the analysis of the research question and the creation of search terms. Readers who are new to SR searching are advised to follow all steps described. More experienced searchers can consider the basic steps to be existing knowledge that will already be part of their normal workflow, although step 4 probably differs from general practice. Experienced searchers will gain the most from reading about the novelties in the method as described in steps 10–13 and comparing the examples given in the supplementary appendix to their own practice.

CREATING A SYSTEMATIC SEARCH STRATEGY

Our methodology for planning and creating a multi-database search strategy consists of the following steps:

• Determine a clear and focused question
• Describe the articles that can answer the question
• Decide which key concepts address the different elements of the question
• Decide which elements should be used for the best results
• Choose an appropriate database and interface to start with
• Document the search process in a text document
• Identify appropriate index terms in the thesaurus of the first database
• Identify synonyms in the thesaurus
• Add variations in search terms
• Use database-appropriate syntax, with parentheses, Boolean operators, and field codes
• Optimize the search
• Evaluate the initial results
• Check for errors
• Translate to other databases
• Test and reiterate

Each step in the process is reflected by an example search described in the supplementary appendix .

1. Determine a clear and focused question

A systematic search can best be applied to a well-defined and precise research or clinical question. Questions that are too broad or too vague cannot be answered easily in a systematic way and will generally result in an overwhelming number of search results. On the other hand, a question that is too specific will result into too few or even zero search results. Various papers describe this process in more detail [ 10 – 12 ].

2. Describe the articles that can answer the question

Although not all clinical or research questions can be answered in the literature, the next step is to presume that the answer can indeed be found in published studies. A good starting point for a search is hypothesizing what the research that can answer the question would look like. These hypothetical (when possible, combined with known) articles can be used as guidance for constructing the search strategy.

3. Decide which key concepts address the different elements of the question

Key concepts are the topics or components that the desired articles should address, such as diseases or conditions, actions, substances, settings, domains (e.g., therapy, diagnosis, etiology), or study types. Key concepts from the research question can be grouped to create elements in the search strategy.

Elements in a search strategy do not necessarily follow the patient, intervention, comparison, outcome (PICO) structure or any other related structure. Using the PICO or another similar framework as guidance can be helpful to consider, especially in the inclusion and exclusion review stage of the SR, but this is not necessary for good search strategy development [ 13 – 15 ]. Sometimes concepts from different parts of the PICO structure can be grouped together into one search element, such as when the desired outcome is frequently described in a certain study type.

4. Decide which elements should be used for the best results

Not all elements of a research question should necessarily be used in the search strategy. Some elements are less important than others or may unnecessarily complicate or restrict a search strategy. Adding an element to a search strategy increases the chance of missing relevant references. Therefore, the number of elements in a search strategy should remain as low as possible to optimize recall.

Using the schema in Figure 1 , elements can be ordered by their specificity and importance to determine the best search approach. Whether an element is more specific or more general can be measured objectively by the number of hits retrieved in a database when searching for a key term representing that element. Depending on the research question, certain elements are more important than others. If articles (hypothetically or known) exist that can answer the question but lack a certain element in their titles, abstracts, or keywords, that element is unimportant to the question. An element can also be unimportant because of expected bias or an overlap with another element.

Schema for determining the optimal order of elements

Bias in elements

The choice of elements in a search strategy can introduce bias through use of overly specific terminology or terms often associated with positive outcomes. For the question “does prolonged breastfeeding improve intelligence outcomes in children?,” searching specifically for the element of duration will introduce bias, as articles that find a positive effect of prolonged breastfeeding will be much more likely to mention time factors in their titles or abstracts.

Overlapping elements

Elements in a question sometimes overlap in their meaning. Sometimes certain therapies are interventions for one specific disease. The Lichtenstein technique, for example, is a repair method for inguinal hernias. There is no need to include an element of “inguinal hernias” to a search for the effectiveness of the Lichtenstein therapy. Likewise, sometimes certain diseases are only found in certain populations. Adding such an overlapping element could lead to missing relevant references.

The elements to use in a search strategy can be found in the plot of elements in Figure 1 , by following the top row from left to right. For this method, we recommend starting with the most important and specific elements. Then, continue with more general and important elements until the number of results is acceptable for screening. Determining how many results are acceptable for screening is often a matter of negotiation with the SR team.

5. Choose an appropriate database and interface to start with

Important factors for choosing databases to use are the coverage and the presence of a thesaurus. For medically oriented searches, the coverage and recall of Embase, which includes the MEDLINE database, are superior to those of MEDLINE [ 16 ]. Each of these two databases has its own thesaurus with its own unique definitions and structure. Because of the complexity of the Embase thesaurus, Emtree, which contains much more specific thesaurus terms than the MEDLINE Medical Subject Headings (MeSH) thesaurus, translation from Emtree to MeSH is easier than the other way around. Therefore, we recommend starting in Embase.

MEDLINE and Embase are available through many different vendors and interfaces. The choice of an interface and primary database is often determined by the searcher’s accessibility. For our method, an interface that allows searching with proximity operators is desirable, and full functionality of the thesaurus, including explosion of narrower terms, is crucial. We recommend developing a personal workflow that always starts with one specific database and interface.

6. Document the search process in a text document

We advise designing and creating the complete search strategies in a log document, instead of directly in the database itself, to register the steps taken and to make searches accountable and reproducible. The developed search strategies can be copied and pasted into the desired databases from the log document. This way, the searcher is in control of the whole process. Any change to the search strategy should be done in the log document, assuring that the search strategy in the log is always the most recent.

7. Identify appropriate index terms in the thesaurus of the first database

Searches should start by identifying appropriate thesaurus terms for the desired elements. The thesaurus of the database is searched for matching index terms for each key concept. We advise restricting the initial terms to the most important and most relevant terms. Later in the process, more general terms can be added in the optimization process, in which the effect on the number of hits, and thus the desirability of adding these terms, can be evaluated more easily.

Several factors can complicate the identification of thesaurus terms. Sometimes, one thesaurus term is found that exactly describes a specific element. In contrast, especially in more general elements, multiple thesaurus terms can be found to describe one element. If no relevant thesaurus terms have been found for an element, free-text terms can be used, and possible thesaurus terms found in the resulting references can be added later (step 11).

Sometimes, no distinct thesaurus term is available for a specific key concept that describes the concept in enough detail. In Emtree, one thesaurus term often combines two or more elements. The easiest solution for combining these terms for a sensitive search is to use such a thesaurus term in all elements where it is relevant. Examples are given in the supplementary appendix .

8. Identify synonyms in the thesaurus

Most thesauri offer a list of synonyms on their term details page (named Synonyms in Emtree and Entry Terms in MeSH). To create a sensitive search strategy for SRs, these terms need to be searched as free-text keywords in the title and abstract fields, in addition to searching their associated thesaurus terms.

The Emtree thesaurus contains more synonyms (300,000) than MeSH does (220,000) [ 17 ]. The difference in number of terms is even higher considering that many synonyms in MeSH are permuted terms (i.e., inversions of phrases using commas).

Thesaurus terms are ordered in a tree structure. When searching for a more general thesaurus term, the more specific (narrower) terms in the branches below that term will also be searched (this is frequently referred to as “exploding” a thesaurus term). However, to perform a sensitive search, all relevant variations of the narrower terms must be searched as free-text keywords in the title or abstract, in addition to relying on the exploded thesaurus term. Thus, all articles that describe a certain narrower topic in their titles and abstracts will already be retrieved before MeSH terms are added.

9. Add variations in search terms (e.g., truncation, spelling differences, abbreviations, opposites)

Truncation allows a searcher to search for words beginning with the same word stem. A search for therap* will, thus, retrieve therapy, therapies, therapeutic, and all other words starting with “therap.” Do not truncate a word stem that is too short. Also, limitations of interfaces should be taken into account, especially in PubMed, where the number of search term variations that can be found by truncation is limited to 600.

Databases contain references to articles using both standard British and American English spellings. Both need to be searched as free-text terms in the title and abstract. Alternatively, many interfaces offer a certain code to replace zero or one characters, allowing a search for “pediatric” or “paediatric” as “p?ediatric.” Table 1 provides a detailed description of the syntax for different interfaces.

Field codes in five most used interfaces for biomedical literature searching

Searching for abbreviations can identify extra, relevant references and retrieve more irrelevant ones. The search can be more focused by combining the abbreviation with an important word that is relevant to its meaning or by using the Boolean “NOT” to exclude frequently observed, clearly irrelevant results. We advise that searchers do not exclude all possible irrelevant meanings, as it is very time consuming to identify all the variations, it will result in unnecessarily complicated search strategies, and it may lead to erroneously narrowing the search and, thereby, reduce recall.

Searching partial abbreviations can be useful for retrieving relevant references. For example, it is very likely that an article would mention osteoarthritis (OA) early in the abstract, replacing all further occurrences of osteoarthritis with OA . Therefore, it may not contain the phrase “hip osteoarthritis” but only “hip oa.”

It is also important to search for the opposites of search terms to avoid bias. When searching for “disease recurrence,” articles about “disease free” may be relevant as well. When the desired outcome is survival , articles about mortality may be relevant.

10. Use database-appropriate syntax, with parentheses, Boolean operators, and field codes

Different interfaces require different syntaxes, the special set of rules and symbols unique to each database that define how a correctly constructed search operates. Common syntax components include the use of parentheses and Boolean operators such as “AND,” “OR,” and “NOT,” which are available in all major interfaces. An overview of different syntaxes for four major interfaces for bibliographic medical databases (PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest) is shown in Table 1 .

Creating the appropriate syntax for each database, in combination with the selected terms as described in steps 7–9, can be challenging. Following the method outlined below simplifies the process:

• Create single-line queries in a text document (not combining multiple record sets), which allows immediate checking of the relevance of retrieved references and efficient optimization.
• Type the syntax (Boolean operators, parentheses, and field codes) before adding terms, which reduces the chance that errors are made in the syntax, especially in the number of parentheses.
• Use predefined proximity structures including parentheses, such as (() ADJ3 ()) in Ovid, that can be reused in the query when necessary.
• Use thesaurus terms separately from free-text terms of each element. Start an element with all thesaurus terms (using “OR”) and follow with the free-text terms. This allows the unique optimization methods as described in step 11.
• When adding terms to an existing search strategy, pay close attention to the position of the cursor. Make sure to place it appropriately either in the thesaurus terms section, in the title/abstract section, or as an addition (broadening) to an existing proximity search.

The supplementary appendix explains the method of building a query in more detail, step by step for different interfaces: PubMed, Ovid, EBSCOhost, Embase.com, and ProQuest. This method results in a basic search strategy designed to retrieve some relevant references upon which a more thorough search strategy can be built with optimization such as described in step 11.

11. Optimize the search

The most important question when performing a systematic search is whether all (or most) potentially relevant articles have been retrieved by the search strategy. This is also the most difficult question to answer, since it is unknown which and how many articles are relevant. It is, therefore, wise first to broaden the initial search strategy, making the search more sensitive, and then check if new relevant articles are found by comparing the set results (i.e., search for Strategy #2 NOT Strategy #1 to see the unique results).

A search strategy should be tested for completeness. Therefore, it is necessary to identify extra, possibly relevant search terms and add them to the test search in an OR relationship with the already used search terms. A good place to start, and a well-known strategy, is scanning the top retrieved articles when sorted by relevance, looking for additional relevant synonyms that could be added to the search strategy.

We have developed a unique optimization method that has not been described before in the literature. This method often adds valuable extra terms to our search strategy and, therefore, extra, relevant references to our search results. Extra synonyms can be found in articles that have been assigned a certain set of thesaurus terms but that lack synonyms in the title and/or abstract that are already present in the current search strategy. Searching for thesaurus terms NOT free-text terms will help identify missed free-text terms in the title or abstract. Searching for free-text terms NOT thesaurus terms will help identify missed thesaurus terms. If this is done repeatedly for each element, leaving the rest of the query unchanged, this method will help add numerous relevant terms to the query. These steps are explained in detail for five different search platforms in the supplementary appendix .

12. Evaluate the initial results

The results should now contain relevant references. If the interface allows relevance ranking, use that in the evaluation. If you know some relevant references that should be included in the research, search for those references specifically; for example, combine a specific (first) author name with a page number and the publication year. Check whether those references are retrieved by the search. If the known relevant references are not retrieved by the search, adapt the search so that they are. If it is unclear which element should be adapted to retrieve a certain article, combine that article with each element separately.

Different outcomes are desired for different types of research questions. For instance, in the case of clinical question answering, the researcher will not be satisfied with many references that contain a lot of irrelevant references. A clinical search should be rather specific and is allowed to miss a relevant reference. In the case of an SR, the researchers do not want to miss any relevant reference and are willing to handle many irrelevant references to do so. The search for references to include in an SR should be very sensitive: no included reference should be missed. A search that is too specific or too sensitive for the intended goal can be adapted to become more sensitive or specific. Steps to increase sensitivity or specificity of a search strategy can be found in the supplementary appendix .

13. Check for errors

Errors might not be easily detected. Sometimes clues can be found in the number of results, either when the number of results is much higher or lower than expected or when many retrieved references are not relevant. However, the number expected is often unknown, and very sensitive search strategies will always retrieve many irrelevant articles. Each query should, therefore, be checked for errors.

One of the most frequently occurring errors is missing the Boolean operator “OR.” When no “OR” is added between two search terms, many interfaces automatically add an “AND,” which unintentionally reduces the number of results and likely misses relevant references. One good strategy to identify missing “OR”s is to go to the web page containing the full search strategy, as translated by the database, and using Ctrl-F search for “AND.” Check whether the occurrences of the “AND” operator are deliberate.

Ideally, search strategies should be checked by other information specialists [ 18 ]. The Peer Review of Electronic Search Strategies (PRESS) checklist offers good guidance for this process [ 4 ]. Apart from the syntax (especially Boolean operators and field codes) of the search strategy, it is wise to have the search terms checked by the clinician or researcher familiar with the topic. At Erasmus MC, researchers and clinicians are involved during the complete process of structuring and optimizing the search strategy. Each word is added after the combined decision of the searcher and the researcher, with the possibility of directly comparing results with and without the new term.

14. Translate to other databases

To retrieve as many relevant references as possible, one has to search multiple databases. Translation of complex and exhaustive queries between different databases can be very time consuming and cumbersome. The single-line search strategy approach detailed above allows quick translations using the find and replace method in Microsoft Word (<Ctrl-H>).

At Erasmus MC, macros based on the find-and-replace method in Microsoft Word have been developed for easy and fast translation between the most used databases for biomedical and health sciences questions. The schema that is followed for the translation between databases is shown in Figure 2 . Most databases simply follow the structure set by the Embase.com search strategy. The translation from Emtree terms to MeSH terms for MEDLINE in Ovid often identifies new terms that need to be added to the Embase.com search strategy before the translation to other databases.

Schematic representation of translation between databases used at Erasmus University Medical Center

Dotted lines represent databases that are used in less than 80% of the searches.

Using five different macros, a thoroughly optimized query in Embase.com can be relatively quickly translated into eight major databases. Basic search strategies will be created to use in many, mostly smaller, databases, because such niche databases often do not have extensive thesauri or advanced syntax options. Also, there is not much need to use extensive syntax because the number of hits and, therefore, the amount of noise in these databases is generally low. In MEDLINE (Ovid), PsycINFO (Ovid), and CINAHL (EBSCOhost), the thesaurus terms must be adapted manually, as each database has its own custom thesaurus. These macros and instructions for their installation, use, and adaptation are available at bit.ly/databasemacros.

15. Test and reiterate

Ideally, exhaustive search strategies should retrieve all references that are covered in a specific database. For SR search strategies, checking searches for their recall is advised. This can be done after included references have been determined by the authors of the systematic review. If additional papers have been identified through other non-database methods (i.e., checking references in included studies), results that were not identified by the database searches should be examined. If these results were available in the databases but not located by the search strategy, the search strategy should be adapted to try to retrieve these results, as they may contain terms that were omitted in the original search strategies. This may enable the identification of additional relevant results.

A methodology for creating exhaustive search strategies has been created that describes all steps of the search process, starting with a question and resulting in thorough search strategies in multiple databases. Many of the steps described are not new, but together, they form a strong method creating high-quality, robust searches in a relatively short time frame.

Our methodology is intended to create thoroughness for literature searches. The optimization method, as described in step 11, will identify missed synonyms or thesaurus terms, unlike any other method that largely depends on predetermined keywords and synonyms. Using this method results in a much quicker search process, compared to traditional methods, especially because of the easier translation between databases and interfaces (step 13). The method is not a guarantee for speed, since speed depends on many factors, including experience. However, by following the steps and using the tools as described above, searchers can gain confidence first and increase speed through practice.

What is new?

This method encourages searchers to start their search development process using empty syntax first and later adding the thesaurus terms and free-text synonyms. We feel this helps the searcher to focus on the search terms, instead of on the structure of the search query. The optimization method in which new terms are found in the already retrieved articles is used in some other institutes as well but has to our knowledge not been described in the literature. The macros to translate search strategies between interfaces are unique in this method.

What is different compared to common practice?

Traditionally, librarians and information specialists have focused on creating complex, multi-line (also called line-by-line) search strategies, consisting of multiple record sets, and this method is frequently advised in the literature and handbooks [ 2 , 19 – 21 ]. Our method, instead, uses single-line searches, which is critical to its success. Single-line search strategies can be easily adapted by adding or dropping a term without having to recode numbers of record sets, which would be necessary in multi-line searches. They can easily be saved in a text document and repeated by copying and pasting for search updates. Single-line search strategies also allow easy translation to other syntaxes using find-and-replace technology to update field codes and other syntax elements or using macros (step 13).

When constructing a search strategy, the searcher might experience that certain parentheses in the syntax are unnecessary, such as parentheses around all search terms in the title/abstract portion, if there is only one such term, there are double parentheses in the proximity statement, or one of the word groups exists for only one word. One might be tempted to omit those parentheses for ease of reading and management. However, during the optimization process, the searcher is likely to find extra synonyms that might consist of one word. To add those terms to the first query (with reduced parentheses) requires adding extra parentheses (meticulously placing and counting them), whereas, in the latter search, it only requires proper placement of those terms.

Many search methods highly depend on the PICO framework. Research states that often PICO or PICOS is not suitable for every question [ 22 , 23 ]. There are other acronyms than PICO—such as sample, phenomenon of interest, design, evaluation, research type (SPIDER) [ 24 ]—but each is just a variant. In our method, the most important and specific elements of a question are being analyzed for building the best search strategy.

Though it is generally recommended that searchers search both MEDLINE and Embase, most use MEDLINE as the starting point. It is considered the gold standard for biomedical searching, partially due to historical reasons, since it was the first of its kind, and more so now that it is freely available via the PubMed interface. Our method can be used with any database as a starting point, but we use Embase instead of MEDLINE or another database for a number of reasons. First, Embase provides both unique content and the complete content of MEDLINE. Therefore, searching Embase will be, by definition, more complete than searching MEDLINE only. Second, the number of terms in Emtree (the Embase thesaurus) is three times as high as that of MeSH (the MEDLINE thesaurus). It is easier to find MeSH terms after all relevant Emtree terms have been identified than to start with MeSH and translate to Emtree.

At Erasmus MC, the researchers sit next to the information specialist during most of the search strategy design process. This way, the researchers can deliver immediate feedback on the relevance of proposed search terms and retrieved references. The search team then combines knowledge about databases with knowledge about the research topic, which is an important condition to create the highest quality searches.

Limitations of the method

One disadvantage of single-line searches compared to multi-line search strategies is that errors are harder to recognize. However, with the methods for optimization as described (step 11), errors are recognized easily because missed synonyms and spelling errors will be identified during the process. Also problematic is that more parentheses are needed, making it more difficult for the searcher and others to assess the logic of the search strategy. However, as parentheses and field codes are typed before the search terms are added (step 10), errors in parentheses can be prevented.

Our methodology works best if used in an interface that allows proximity searching. It is recommended that searchers with access to an interface with proximity searching capabilities select one of those as the initial database to develop and optimize the search strategy. Because the PubMed interface does not allow proximity searches, phrases or Boolean “AND” combinations are required. Phrase searching complicates the process and is more specific, with the higher risk of missing relevant articles, and using Boolean “AND” combinations increases sensitivity but at an often high loss of specificity. Due to some searchers’ lack of access to expensive databases or interfaces, the freely available PubMed interface may be necessary to use, though it should never be the sole database used for an SR [ 2 , 16 , 25 ]. A limitation of our method is that it works best with subscription-based and licensed resources.

Another limitation is the customization of the macros to a specific institution’s resources. The macros for the translation between different database interfaces only work between the interfaces as described. To mitigate this, we recommend using the find-and-replace functionality of text editors like Microsoft Word to ease the translation of syntaxes between other databases. Depending on one’s institutional resources, custom macros can be developed using similar methods.

Results of the method

Whether this method results in exhaustive searches where no important article is missed is difficult to determine, because the number of relevant articles is unknown for any topic. A comparison of several parameters of 73 published reviews that were based on a search developed with this method to 258 reviews that acknowledged information specialists from other Dutch academic hospitals shows that the performance of the searches following our method is comparable to those performed in other institutes but that the time needed to develop the search strategies was much shorter than the time reported for the other reviews [ 9 ].

CONCLUSIONS

With the described method, searchers can gain confidence in their search strategies by finding many relevant words and creating exhaustive search strategies quickly. The approach can be used when performing SR searches or for other purposes such as answering clinical questions, with different expectations of the search’s precision and recall. This method, with practice, provides a stepwise approach that facilitates the search strategy development process from question clarification to final iteration and beyond.

SUPPLEMENTAL FILE

Acknowledgments.

We highly appreciate the work that was done by our former colleague Louis Volkers, who in his twenty years as an information specialist in Erasmus MC laid the basis for our method. We thank Professor Oscar Franco for reviewing earlier drafts of this article.