how to identify themes in literature review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Do Thematic Analysis | Step-by-Step Guide & Examples

How to Do Thematic Analysis | Step-by-Step Guide & Examples

Published on September 6, 2019 by Jack Caulfield . Revised on June 22, 2023.

Thematic analysis is a method of analyzing qualitative data . It is usually applied to a set of texts, such as an interview or transcripts . The researcher closely examines the data to identify common themes – topics, ideas and patterns of meaning that come up repeatedly.

There are various approaches to conducting thematic analysis, but the most common form follows a six-step process: familiarization, coding, generating themes, reviewing themes, defining and naming themes, and writing up. Following this process can also help you avoid confirmation bias when formulating your analysis.

This process was originally developed for psychology research by Virginia Braun and Victoria Clarke . However, thematic analysis is a flexible method that can be adapted to many different kinds of research.

Table of contents

When to use thematic analysis, different approaches to thematic analysis, step 1: familiarization, step 2: coding, step 3: generating themes, step 4: reviewing themes, step 5: defining and naming themes, step 6: writing up, other interesting articles.

Thematic analysis is a good approach to research where you’re trying to find out something about people’s views, opinions, knowledge, experiences or values from a set of qualitative data – for example, interview transcripts , social media profiles, or survey responses .

Some types of research questions you might use thematic analysis to answer:

• How do patients perceive doctors in a hospital setting?
• What are young women’s experiences on dating sites?
• What are non-experts’ ideas and opinions about climate change?
• How is gender constructed in high school history teaching?

To answer any of these questions, you would collect data from a group of relevant participants and then analyze it. Thematic analysis allows you a lot of flexibility in interpreting the data, and allows you to approach large data sets more easily by sorting them into broad themes.

However, it also involves the risk of missing nuances in the data. Thematic analysis is often quite subjective and relies on the researcher’s judgement, so you have to reflect carefully on your own choices and interpretations.

Pay close attention to the data to ensure that you’re not picking up on things that are not there – or obscuring things that are.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Once you’ve decided to use thematic analysis, there are different approaches to consider.

There’s the distinction between inductive and deductive approaches:

• An inductive approach involves allowing the data to determine your themes.
• A deductive approach involves coming to the data with some preconceived themes you expect to find reflected there, based on theory or existing knowledge.

Ask yourself: Does my theoretical framework give me a strong idea of what kind of themes I expect to find in the data (deductive), or am I planning to develop my own framework based on what I find (inductive)?

There’s also the distinction between a semantic and a latent approach:

• A semantic approach involves analyzing the explicit content of the data.
• A latent approach involves reading into the subtext and assumptions underlying the data.

Ask yourself: Am I interested in people’s stated opinions (semantic) or in what their statements reveal about their assumptions and social context (latent)?

After you’ve decided thematic analysis is the right method for analyzing your data, and you’ve thought about the approach you’re going to take, you can follow the six steps developed by Braun and Clarke .

The first step is to get to know our data. It’s important to get a thorough overview of all the data we collected before we start analyzing individual items.

This might involve transcribing audio , reading through the text and taking initial notes, and generally looking through the data to get familiar with it.

Next up, we need to code the data. Coding means highlighting sections of our text – usually phrases or sentences – and coming up with shorthand labels or “codes” to describe their content.

Let’s take a short example text. Say we’re researching perceptions of climate change among conservative voters aged 50 and up, and we have collected data through a series of interviews. An extract from one interview looks like this:

In this extract, we’ve highlighted various phrases in different colors corresponding to different codes. Each code describes the idea or feeling expressed in that part of the text.

At this stage, we want to be thorough: we go through the transcript of every interview and highlight everything that jumps out as relevant or potentially interesting. As well as highlighting all the phrases and sentences that match these codes, we can keep adding new codes as we go through the text.

After we’ve been through the text, we collate together all the data into groups identified by code. These codes allow us to gain a a condensed overview of the main points and common meanings that recur throughout the data.

Prevent plagiarism. Run a free check.

Next, we look over the codes we’ve created, identify patterns among them, and start coming up with themes.

Themes are generally broader than codes. Most of the time, you’ll combine several codes into a single theme. In our example, we might start combining codes into themes like this:

At this stage, we might decide that some of our codes are too vague or not relevant enough (for example, because they don’t appear very often in the data), so they can be discarded.

Other codes might become themes in their own right. In our example, we decided that the code “uncertainty” made sense as a theme, with some other codes incorporated into it.

Again, what we decide will vary according to what we’re trying to find out. We want to create potential themes that tell us something helpful about the data for our purposes.

Now we have to make sure that our themes are useful and accurate representations of the data. Here, we return to the data set and compare our themes against it. Are we missing anything? Are these themes really present in the data? What can we change to make our themes work better?

If we encounter problems with our themes, we might split them up, combine them, discard them or create new ones: whatever makes them more useful and accurate.

For example, we might decide upon looking through the data that “changing terminology” fits better under the “uncertainty” theme than under “distrust of experts,” since the data labelled with this code involves confusion, not necessarily distrust.

Now that you have a final list of themes, it’s time to name and define each of them.

Defining themes involves formulating exactly what we mean by each theme and figuring out how it helps us understand the data.

Naming themes involves coming up with a succinct and easily understandable name for each theme.

For example, we might look at “distrust of experts” and determine exactly who we mean by “experts” in this theme. We might decide that a better name for the theme is “distrust of authority” or “conspiracy thinking”.

Finally, we’ll write up our analysis of the data. Like all academic texts, writing up a thematic analysis requires an introduction to establish our research question, aims and approach.

We should also include a methodology section, describing how we collected the data (e.g. through semi-structured interviews or open-ended survey questions ) and explaining how we conducted the thematic analysis itself.

The results or findings section usually addresses each theme in turn. We describe how often the themes come up and what they mean, including examples from the data as evidence. Finally, our conclusion explains the main takeaways and shows how the analysis has answered our research question.

In our example, we might argue that conspiracy thinking about climate change is widespread among older conservative voters, point out the uncertainty with which many voters view the issue, and discuss the role of misinformation in respondents’ perceptions.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Discourse analysis
• Cohort study
• Peer review
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Caulfield, J. (2023, June 22). How to Do Thematic Analysis | Step-by-Step Guide & Examples. Scribbr. Retrieved April 15, 2024, from https://www.scribbr.com/methodology/thematic-analysis/

Is this article helpful?

Jack Caulfield

Other students also liked, what is qualitative research | methods & examples, inductive vs. deductive research approach | steps & examples, critical discourse analysis | definition, guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism, run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 15 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

How To Structure Your Literature Review

3 options to help structure your chapter.

By: Amy Rommelspacher (PhD) | Reviewer: Dr Eunice Rautenbach | November 2020 (Updated May 2023)

Writing the literature review chapter can seem pretty daunting when you’re piecing together your dissertation or thesis. As  we’ve discussed before , a good literature review needs to achieve a few very important objectives – it should:

• Demonstrate your knowledge of the research topic
• Identify the gaps in the literature and show how your research links to these
• Provide the foundation for your conceptual framework (if you have one)
• Inform your own  methodology and research design

To achieve this, your literature review needs a well-thought-out structure . Get the structure of your literature review chapter wrong and you’ll struggle to achieve these objectives. Don’t worry though – in this post, we’ll look at how to structure your literature review for maximum impact (and marks!).

But wait – is this the right time?

Deciding on the structure of your literature review should come towards the end of the literature review process – after you have collected and digested the literature, but before you start writing the chapter. 

In other words, you need to first develop a rich understanding of the literature before you even attempt to map out a structure. There’s no use trying to develop a structure before you’ve fully wrapped your head around the existing research.

Equally importantly, you need to have a structure in place before you start writing , or your literature review will most likely end up a rambling, disjointed mess. 

Importantly, don’t feel that once you’ve defined a structure you can’t iterate on it. It’s perfectly natural to adjust as you engage in the writing process. As we’ve discussed before , writing is a way of developing your thinking, so it’s quite common for your thinking to change – and therefore, for your chapter structure to change – as you write. 

Need a helping hand?

Like any other chapter in your thesis or dissertation, your literature review needs to have a clear, logical structure. At a minimum, it should have three essential components – an  introduction , a  body   and a  conclusion . 

Let’s take a closer look at each of these.

1: The Introduction Section

Just like any good introduction, the introduction section of your literature review should introduce the purpose and layout (organisation) of the chapter. In other words, your introduction needs to give the reader a taste of what’s to come, and how you’re going to lay that out. Essentially, you should provide the reader with a high-level roadmap of your chapter to give them a taste of the journey that lies ahead.

Here’s an example of the layout visualised in a literature review introduction:

Your introduction should also outline your topic (including any tricky terminology or jargon) and provide an explanation of the scope of your literature review – in other words, what you  will   and  won’t   be covering (the delimitations ). This helps ringfence your review and achieve a clear focus . The clearer and narrower your focus, the deeper you can dive into the topic (which is typically where the magic lies). 

Depending on the nature of your project, you could also present your stance or point of view at this stage. In other words, after grappling with the literature you’ll have an opinion about what the trends and concerns are in the field as well as what’s lacking. The introduction section can then present these ideas so that it is clear to examiners that you’re aware of how your research connects with existing knowledge .

2: The Body Section

The body of your literature review is the centre of your work. This is where you’ll present, analyse, evaluate and synthesise the existing research. In other words, this is where you’re going to earn (or lose) the most marks. Therefore, it’s important to carefully think about how you will organise your discussion to present it in a clear way. 

The body of your literature review should do just as the description of this chapter suggests. It should “review” the literature – in other words, identify, analyse, and synthesise it. So, when thinking about structuring your literature review, you need to think about which structural approach will provide the best “review” for your specific type of research and objectives (we’ll get to this shortly).

There are (broadly speaking)  three options  for organising your literature review.

Option 1: Chronological (according to date)

Organising the literature chronologically is one of the simplest ways to structure your literature review. You start with what was published first and work your way through the literature until you reach the work published most recently. Pretty straightforward.

The benefit of this option is that it makes it easy to discuss the developments and debates in the field as they emerged over time. Organising your literature chronologically also allows you to highlight how specific articles or pieces of work might have changed the course of the field – in other words, which research has had the most impact . Therefore, this approach is very useful when your research is aimed at understanding how the topic has unfolded over time and is often used by scholars in the field of history. That said, this approach can be utilised by anyone that wants to explore change over time .

For example , if a student of politics is investigating how the understanding of democracy has evolved over time, they could use the chronological approach to provide a narrative that demonstrates how this understanding has changed through the ages.

Here are some questions you can ask yourself to help you structure your literature review chronologically.

• What is the earliest literature published relating to this topic?
• How has the field changed over time? Why?
• What are the most recent discoveries/theories?

In some ways, chronology plays a part whichever way you decide to structure your literature review, because you will always, to a certain extent, be analysing how the literature has developed. However, with the chronological approach, the emphasis is very firmly on how the discussion has evolved over time , as opposed to how all the literature links together (which we’ll discuss next ).

Option 2: Thematic (grouped by theme)

The thematic approach to structuring a literature review means organising your literature by theme or category – for example, by independent variables (i.e. factors that have an impact on a specific outcome).

As you’ve been collecting and synthesising literature , you’ll likely have started seeing some themes or patterns emerging. You can then use these themes or patterns as a structure for your body discussion. The thematic approach is the most common approach and is useful for structuring literature reviews in most fields.

For example, if you were researching which factors contributed towards people trusting an organisation, you might find themes such as consumers’ perceptions of an organisation’s competence, benevolence and integrity. Structuring your literature review thematically would mean structuring your literature review’s body section to discuss each of these themes, one section at a time.

Here are some questions to ask yourself when structuring your literature review by themes:

• Are there any patterns that have come to light in the literature?
• What are the central themes and categories used by the researchers?
• Do I have enough evidence of these themes?

PS – you can see an example of a thematically structured literature review in our literature review sample walkthrough video here.

Option 3: Methodological

The methodological option is a way of structuring your literature review by the research methodologies used . In other words, organising your discussion based on the angle from which each piece of research was approached – for example, qualitative , quantitative or mixed  methodologies.

Structuring your literature review by methodology can be useful if you are drawing research from a variety of disciplines and are critiquing different methodologies. The point of this approach is to question  how  existing research has been conducted, as opposed to  what  the conclusions and/or findings the research were.

For example, a sociologist might centre their research around critiquing specific fieldwork practices. Their literature review will then be a summary of the fieldwork methodologies used by different studies.

Here are some questions you can ask yourself when structuring your literature review according to methodology:

• Which methodologies have been utilised in this field?
• Which methodology is the most popular (and why)?
• What are the strengths and weaknesses of the various methodologies?
• How can the existing methodologies inform my own methodology?

3: The Conclusion Section

Once you’ve completed the body section of your literature review using one of the structural approaches we discussed above, you’ll need to “wrap up” your literature review and pull all the pieces together to set the direction for the rest of your dissertation or thesis.

The conclusion is where you’ll present the key findings of your literature review. In this section, you should emphasise the research that is especially important to your research questions and highlight the gaps that exist in the literature. Based on this, you need to make it clear what you will add to the literature – in other words, justify your own research by showing how it will help fill one or more of the gaps you just identified.

Last but not least, if it’s your intention to develop a conceptual framework for your dissertation or thesis, the conclusion section is a good place to present this.

Example: Thematically Structured Review

In the video below, we unpack a literature review chapter so that you can see an example of a thematically structure review in practice.

Let’s Recap

In this article, we’ve  discussed how to structure your literature review for maximum impact. Here’s a quick recap of what  you need to keep in mind when deciding on your literature review structure:

• Just like other chapters, your literature review needs a clear introduction , body and conclusion .
• The introduction section should provide an overview of what you will discuss in your literature review.
• The body section of your literature review can be organised by chronology , theme or methodology . The right structural approach depends on what you’re trying to achieve with your research.
• The conclusion section should draw together the key findings of your literature review and link them to your research questions.

If you’re ready to get started, be sure to download our free literature review template to fast-track your chapter outline.

Psst… there’s more!

This post is an extract from our bestselling Udemy Course, Literature Review Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

27 Comments

Great work. This is exactly what I was looking for and helps a lot together with your previous post on literature review. One last thing is missing: a link to a great literature chapter of an journal article (maybe with comments of the different sections in this review chapter). Do you know any great literature review chapters?

I agree with you Marin… A great piece

I agree with Marin. This would be quite helpful if you annotate a nicely structured literature from previously published research articles.

Awesome article for my research.

I thank you immensely for this wonderful guide

It is indeed thought and supportive work for the futurist researcher and students

Very educative and good time to get guide. Thank you

Great work, very insightful. Thank you.

Thanks for this wonderful presentation. My question is that do I put all the variables into a single conceptual framework or each hypothesis will have it own conceptual framework?

Thank you very much, very helpful

This is very educative and precise . Thank you very much for dropping this kind of write up .

Pheeww, so damn helpful, thank you for this informative piece.

I’m doing a research project topic ; stool analysis for parasitic worm (enteric) worm, how do I structure it, thanks.

comprehensive explanation. Help us by pasting the URL of some good “literature review” for better understanding.

great piece. thanks for the awesome explanation. it is really worth sharing. I have a little question, if anyone can help me out, which of the options in the body of literature can be best fit if you are writing an architectural thesis that deals with design?

I am doing a research on nanofluids how can l structure it?

Beautifully clear.nThank you!

Lucid! Thankyou!

Brilliant work, well understood, many thanks

I like how this was so clear with simple language 😊😊 thank you so much 😊 for these information 😊

Insightful. I was struggling to come up with a sensible literature review but this has been really helpful. Thank you!

You have given thought-provoking information about the review of the literature.

Thank you. It has made my own research better and to impart your work to students I teach

I learnt a lot from this teaching. It’s a great piece.

I am doing research on EFL teacher motivation for his/her job. How Can I structure it? Is there any detailed template, additional to this?

You are so cool! I do not think I’ve read through something like this before. So nice to find somebody with some genuine thoughts on this issue. Seriously.. thank you for starting this up. This site is one thing that is required on the internet, someone with a little originality!

I’m asked to do conceptual, theoretical and empirical literature, and i just don’t know how to structure it

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Research Process :: Step by Step

• Introduction
• Select Topic
• Identify Keywords
• Background Information
• Develop Research Questions
• Refine Topic
• Search Strategy
• Popular Databases
• Evaluate Sources
• Types of Periodicals
• Reading Scholarly Articles
• Primary & Secondary Sources
• Organize / Take Notes
• Writing & Grammar Resources
• Annotated Bibliography
• Literature Review
• Citation Styles
• Paraphrasing
• Privacy / Confidentiality
• Research Process
• Selecting Your Topic
• Identifying Keywords
• Gathering Background Info
• Evaluating Sources

Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.  

What is a literature review?

A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries

A literature review must do these things:

• be organized around and related directly to the thesis or research question you are developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

Ask yourself questions like these:

• What is the specific thesis, problem, or research question that my literature review helps to define?
• What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
• What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
• How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
• Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
• Have I cited and discussed studies contrary to my perspective?
• Will the reader find my literature review relevant, appropriate, and useful?

Ask yourself questions like these about each book or article you include:

• Has the author formulated a problem/issue?
• Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
• Could the problem have been approached more effectively from another perspective?
• What is the author's research orientation (e.g., interpretive, critical science, combination)?
• What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
• What is the relationship between the theoretical and research perspectives?
• Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
• In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
• In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
• How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
• In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
• How does this book or article relate to the specific thesis or question I am developing?

Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

• << Previous: Annotated Bibliography
• Next: Step 5: Cite Sources >>
• Last Updated: Apr 10, 2024 9:23 AM
• URL: https://libguides.uta.edu/researchprocess

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

• 7. Write a Literature Review
• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate
• 6. Synthesize

Write a Literature Review

Writing support on campus, need more help.

Photo Credit: UO Libraries

Some points to remember

• Include only the most important points from each source -- you want to digest, not quote from, the sources.
• The value of the review for you audience will consist in a clear, well-organized synopsis of what has been found so far on your topic. 
• Avoid plagiarism in your lit review. Consult this UO Libraries tutorial on Academic Integrity if you need some guidance.

If you would like more pointers about how to approach your literature review, this this handout from The Writing Center at UNC-Chapel Hill  suggests several effective strategies.

From UNC-Chapel Hill  and  University of Toronto

Note : Please check the websites below for availability of online or remote services:

• Writing Lab Service Customized academic assistance for all international students
• How to Write a Literature Review (UO Libraries tutorial)
• Exploring Academic Integrity in Your Research

If you have questions related to a field or discipline, consider reaching out to a Subject Librarian by email, phone, or by scheduling an appointment for a free consultation:

• Subject Librarians
• << Previous: 6. Synthesize
• Last Updated: Jan 10, 2024 4:46 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

Search form

You are here, identifying themes to structure your literature review.

In order to develop the sections in your literature review you will need to be able to draw out the key themes from your reading. The following questions are designed to help you achieve this:

• What do I already know?
• What are the key concepts and definitions within this area of study?
• What are the political standpoints and historical context?
• What are the beliefs underpinning this area of study?
• Why is it important to study this research problem and how does it relate to the current national/international priorities within education?
• What are the key questions asked by other researchers in this area and how can these help me to develop my own research focus and questions?
• What research strategies and methods have been used by other researchers in this area of study?
• Can I learn from mistakes that have been made by other researchers?
• What are the key theoretical perspectives in this area?
• What are the key research findings?
• How do these existing ideas from theory and research link together?
• How can the literature help me to interpret my findings? E.g. Are there analytical frameworks that have been used by other researchers that I could use to analyse my findings?
• Are there any conflicting research findings or areas of controversy?
• Are there any gaps in the existing research?

(Adapted from Langdridge and Hagger-Johnson (2009); Bryman (2008); and Hart, cited in Punch, 2009)

• research methods
• literature reviews

Creative Commons

Research Methods and Design

• Action Research
• Case Study Design

Literature Review

• Quantitative Research Methods
• Qualitative Research Methods
• Mixed Methods Study
• Indigenous Research and Ethics This link opens in a new window
• Identifying Empirical Research Articles This link opens in a new window
• Research Ethics and Quality
• Data Literacy
• Get Help with Writing Assignments

A literature review is a discussion of the literature (aka. the "research" or "scholarship") surrounding a certain topic. A good literature review doesn't simply summarize the existing material, but provides thoughtful synthesis and analysis. The purpose of a literature review is to orient your own work within an existing body of knowledge. A literature review may be written as a standalone piece or be included in a larger body of work.

You can read more about literature reviews, what they entail, and how to write one, using the resources below. 

Am I the only one struggling to write a literature review?

Dr. Zina O'Leary explains the misconceptions and struggles students often have with writing a literature review. She also provides step-by-step guidance on writing a persuasive literature review.

An Introduction to Literature Reviews

Dr. Eric Jensen, Professor of Sociology at the University of Warwick, and Dr. Charles Laurie, Director of Research at Verisk Maplecroft, explain how to write a literature review, and why researchers need to do so. Literature reviews can be stand-alone research or part of a larger project. They communicate the state of academic knowledge on a given topic, specifically detailing what is still unknown.

This is the first video in a whole series about literature reviews. You can find the rest of the series in our SAGE database, Research Methods:

Videos covering research methods and statistics

To login from SAGE, click Institution, then Access via Your Institution, then find and select City University of Seattle

Identify Themes and Gaps in Literature (with real examples) | Scribbr

Finding connections between sources is key to organizing the arguments and structure of a good literature review. In this video, you'll learn how to identify themes, debates, and gaps between sources, using examples from real papers.

4 Tips for Writing a Literature Review's Intro, Body, and Conclusion | Scribbr

While each review will be unique in its structure--based on both the existing body of both literature and the overall goals of your own paper, dissertation, or research--this video from Scribbr does a good job simplifying the goals of writing a literature review for those who are new to the process. In this video, you’ll learn what to include in each section, as well as 4 tips for the main body illustrated with an example.

• Literature Review This chapter in SAGE's Encyclopedia of Research Design describes the types of literature reviews and scientific standards for conducting literature reviews.
• UNC Writing Center: Literature Reviews This handout from the Writing Center at UNC will explain what literature reviews are and offer insights into the form and construction of literature reviews in the humanities, social sciences, and sciences.
• Purdue OWL: Writing a Literature Review The overview of literature reviews comes from Purdue's Online Writing Lab. It explains the basic why, what, and how of writing a literature review.

Organizational Tools for Literature Reviews

One of the most daunting aspects of writing a literature review is organizing your research. There are a variety of strategies that you can use to help you in this task. We've highlighted just a few ways writers keep track of all that information! You can use a combination of these tools or come up with your own organizational process. The key is choosing something that works with your own learning style.

Citation Managers

Citation managers are great tools, in general, for organizing research, but can be especially helpful when writing a literature review. You can keep all of your research in one place, take notes, and organize your materials into different folders or categories. Read more about citations managers here:

• Manage Citations & Sources

Concept Mapping

Some writers use concept mapping (sometimes called flow or bubble charts or "mind maps") to help them visualize the ways in which the research they found connects.

There is no right or wrong way to make a concept map. There are a variety of online tools that can help you create a concept map or you can simply put pen to paper. To read more about concept mapping, take a look at the following help guides:

• Using Concept Maps From Williams College's guide, Literature Review: A Self-guided Tutorial

Synthesis Matrix

A synthesis matrix is is a chart you can use to help you organize your research into thematic categories. By organizing your research into a matrix, like the examples below, can help you visualize the ways in which your sources connect. 

• Walden University Writing Center: Literature Review Matrix Find a variety of literature review matrix examples and templates from Walden University.
• Writing A Literature Review and Using a Synthesis Matrix An example synthesis matrix created by NC State University Writing and Speaking Tutorial Service Tutors. If you would like a copy of this synthesis matrix in a different format, like a Word document, please ask a librarian. CC-BY-SA 3.0
• << Previous: Case Study Design
• Next: Quantitative Research Methods >>
• Last Updated: Feb 6, 2024 9:20 AM

CityU Home - CityU Catalog

Literature Reviews

• 5. Synthesize your findings
• Getting started
• Types of reviews
• 1. Define your research question
• 2. Plan your search
• 3. Search the literature
• 4. Organize your results

How to synthesize

Approaches to synthesis.

• 6. Write the review
• Artificial intelligence (AI) tools
• Thompson Writing Studio This link opens in a new window
• Need to write a systematic review? This link opens in a new window

Contact a Librarian

Ask a Librarian

In the synthesis step of a literature review, researchers analyze and integrate information from selected sources to identify patterns and themes. This involves critically evaluating findings, recognizing commonalities, and constructing a cohesive narrative that contributes to the understanding of the research topic.

Here are some examples of how to approach synthesizing the literature:

💡 By themes or concepts

🕘 Historically or chronologically

📊 By methodology

These organizational approaches can also be used when writing your review. It can be beneficial to begin organizing your references by these approaches in your citation manager by using folders, groups, or collections.

Create a synthesis matrix

A synthesis matrix allows you to visually organize your literature.

Topic: ______________________________________________

Topic: Chemical exposure to workers in nail salons

• << Previous: 4. Organize your results
• Next: 6. Write the review >>
• Last Updated: Apr 3, 2024 12:40 PM
• URL: https://guides.library.duke.edu/lit-reviews

Services for...

• Faculty & Instructors
• Graduate Students
• Undergraduate Students
• International Students
• Patrons with Disabilities

• Harmful Language Statement
• Re-use & Attribution / Privacy
• Support the Libraries

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE : Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: Apr 15, 2024 12:53 PM
• URL: https://libguides.usc.edu/writingguide

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Where instructors and editors talk writing.

• Literature Review Essentials: Identify Themes

Theme 1: Name or description of theme, which will eventually be made into a topic sentence Author Name (Year), Paraphrase of relevant material related to the theme Author Name (Year), Paraphrase of relevant material related to the theme Repeat as needed for all the sources you’ve found that deal with this theme Theme 2: Name or description of theme, which will eventually be made into a topic sentence Author Name (Year), Paraphrase of relevant material related to the theme Author Name (Year), Paraphrase of relevant material related to the theme Repeat as needed for all the sources you’ve found that deal with this theme Continue as needed, depending on how many themes you have identified in the literature and how many sources have information to contribute to the themes.

26 comments :

This was very helpful Jes! Thanks a lot.

We're glad you found this post helpful, Eniale!

Thank you Jes. I found your method to thematic literature review very useful and easy to understand. I am a Phd student in Education with the University of Durban.I am based in Zimbabwe.

We are glad you found this method useful!

I am having difficulties locating an example of a completed literature review outline. Where would I find additional examples? Thank you.

You’re welcome!

Hi is literature review writing same as writing a thematic summary after reading articles based on the given topic?

Great question! It depends on the assignment (what faculty want with the thematic summary assignment). For literature reviews, paragraphs revolve around themes within the literature as opposed to authors. So, when Jes uses the phrasing, "thematic literature reviews," she is referring to how literature reviews are about synthesizing themes across the literature as opposed to creating paragraphs that are summaries for each singular author/ source. You can read more about synthesis on our website http://bit.ly/2JEUK09

thank you, this one is useful.

I've been in a slump for weeks regarding a literature review assignment. This is very very helpful! Thank you!

We're happy this helped! Good luck with your literature review!

I'm doing a lit review for a post grad certificate and I have to discuss the findings using a framework analysis, can you lead me to the right section. Yours struggling

Hello, You might check with your professor regarding what is meant by “framework analysis” in relation to the finding of your literature review. While the Writing Center has some sources on components of writing literature reviews, it is best to speak with faculty regarding assignment specifics. You might also check out our page on literature reviews: https://academicguides.waldenu.edu/writingcenter/assignments/literaturereview We hope this helps!

i have found it useful, it has polished me some how, my problem in the literature review is how to formulate themes and subthemes for the topic. forexample dealing with solid waste skips monitoring and management for municpalities. what themes can i build on in the study. thank you dear.

Hello! I think creating an outline my help you identify some main themes and sub themes and then you can choose what themes and sub themes you want to focus on from there. You might check out our outlining page for tips on creating an outline: http://bit.ly/2WP3UCe. We hope this helps!

Thank you! My professor referenced this link/article for the literature review assignment in a research class. This was helpful.

So glad you found it helpful! Thank you for commenting!

This really was helpful. I feel so much better about tackling my lit review now. Thanks!

Helping writers gain confidence in their skills is one of our primary goals for this blog. Thank you for reaching out with this kind note to let us know we are on the right track! :)

Really informative study. It helped me in finding themes in literature. Plz if u can give me your email. It will be very helpful to me in writing my dissertation

This was very helpful; clear and concise. Thank you!

Oh Jes! you just sent me back to the drawing board, guilty of annotated bibliography. Thank you for this helpful writeup.

I'm sure you're not alone! And there is a reason this is a blog post topic. :) Kudos for identifying potential issues in your own writing and thank you so much for reading and commenting!

im failing to understand the concept themes

This can be a tricky concept to grasp, which is why we've created blog posts like this one. Basically, "themes" are ideas that connect so that writers are organizing based on themes rather than on sources. So instead of having one paragraph about one article, another paragraph about a different article, and a third paragraph about a third article, the paragraphs would be organized based on common ideas shared between two or three of the articles. You might check out our MEAL plan resources (https://academicanswers.waldenu.edu/faq/72800), or our resources for synthesis (https://academicanswers.waldenu.edu/faq/72694) for further clarification. Hope this helps! Thanks for reading!

Search the Blog

Featured posts.

• Expert Advice
• Writing Center Services
• Scholarly Writing
• Capstone Writing
• Grammar and Mechanics
• Writer's Workshop
• Social Change
• Literature Review
• International/Multilingual Students
• Passive Voice

WriteCast Podcast

Visit the Writing Center's Website

• About the Blog

Blog Archive

• ►  May (1)
• ►  April (1)
• ►  February (1)
• ►  January (1)
• ►  December (1)
• ►  November (1)
• ►  September (1)
• ►  July (1)
• ►  June (1)
• ►  March (2)
• ►  October (2)
• ►  September (2)
• ►  August (2)
• ►  July (2)
• ►  June (2)
• ►  April (2)
• ►  March (3)
• ►  January (3)
• ►  November (3)
• ►  October (3)
• ►  September (4)
• ►  August (4)
• ►  July (5)
• ►  June (5)
• ►  May (5)
• ►  April (9)
• ►  March (7)
• ►  February (8)
• ►  January (9)
• ►  December (9)
• ►  November (9)
• ►  October (9)
• ►  September (8)
• ►  August (7)
• ►  July (8)
• ►  June (8)
• ►  May (7)
• ►  March (9)
• ►  December (8)
• ►  August (9)
• ►  July (6)
• ►  June (6)
• ►  May (9)
• ►  April (8)
• Webinar Update: April Webinars!
• Literature Review Essentials: Align Problems
• Relaxation Reminder
• Literature Review Essentials: Construct Paragraphs
• WriteCast Episode 36: Social Change and Difficult ...
• Literature Review Essentials: A Five-Part Blog Series
• Literature Review Essentials: Curate Information
• March Webinar Update
• ►  November (8)
• ►  September (9)
• ►  June (9)
• ►  February (9)
• ►  June (4)
• ►  May (4)
• ►  April (5)
• ►  March (6)
• ►  February (5)
• ►  January (5)
• ►  December (6)
• ►  November (5)
• ►  October (5)
• ►  September (5)
• ►  August (5)
• ►  April (7)
• ►  February (4)
• ►  December (4)
• ►  November (4)
• ►  October (7)
• ►  July (7)
• ►  April (6)
• ►  March (4)
• ►  September (7)
• ►  August (6)
• ►  June (3)
• ►  January (6)
• ►  October (6)
• ►  September (3)
• ►  April (4)
• ►  February (2)
• ►  January (2)
• ►  December (2)
• ►  November (2)
• ►  August (3)

© Walden University Writing Center SoraTemplates . Posts RSS . Comments RSS

How to Identify the Theme in a Literary Work

Juan Paz / EyeEm / Getty Images

• Study Guides
• Authors & Texts
• Top Picks Lists
• Best Sellers
• Plays & Drama
• Shakespeare
• Short Stories
• Children's Books
• M.A., English Literature, California State University - Sacramento
• B.A., English, California State University - Sacramento

A theme is a central or underlying idea in literature , which may be stated directly or indirectly. All novels , stories, poems, and other literary works have at least one theme running through them. The writer may express insight about humanity or a worldview through a theme.

Subject Versus Theme

Don't confuse the subject of a work with its theme:

• The subject is a topic that acts as the foundation for a work of literature, such as marriage in 19th-century France.
• A  theme is an opinion the author expresses on the subject, for instance, the author's dissatisfaction with the narrow confines of French bourgeois marriage during that period.

Major and Minor Themes

There can be major and minor themes in works of literature:

• A major theme is an idea that a writer repeats in his work, making it the most significant idea in a literary work.
• A minor theme, on the other hand, refers to an idea that appears in a work briefly and that may or may not give way to another minor theme.

Read and Analyze the Work

Before you attempt to identify the theme of a work, you must have read the work, and you should understand at least the basics of the plot , characterizations, and other literary elements. Spend some time thinking about the main subjects covered in work. Common subjects include coming of age, death and mourning, racism, beauty, heartbreak and betrayal, loss of innocence, and power and corruption.

Next, consider what the author's view on these subjects might be. These views will point you toward the work's themes. Here's how to get started.

How to Identify Themes in a Published Work

• Note the plot of the work: Take a few moments to write down the main literary elements: plot, characterization, setting, tone, language style, etc. What were the conflicts in the work? What was the most important moment in the work? Does the author resolve the conflict? How did the work end?
• Identify the subject of the work: If you were to tell a friend what the work of literature was about, how would you describe that? What would you say is the topic?
• Who is the protagonist (the main character)?  How does he or she change? Does the protagonist affect other characters? How does this character relate to others?
• Assess the author's point of view : Finally, determine the author's view toward the characters and the choices they make. What might be the author's attitude toward the resolution of the main conflict? What message might the author be sending us? This message is the theme. You may find clues in the language used, in quotes from main characters, or in the final resolution of the conflicts.

Note that none of these elements (plot, subject, character, or point of view ) constitute a theme in and of itself. But identifying them is an important first step in identifying a work's major theme or themes.

• How to Find the Theme of a Book or Short Story
• Use a Concept Map for Your Literature Midterms and Finals
• 10 Common Themes in Literature
• How to Summarize a Plot
• How a Narrative Arc Structures a Story
• Writing About Literature: Ten Sample Topics for Comparison & Contrast Essays
• Stylistics and Elements of Style in Literature
• What Literature Can Teach Us
• How to Write a Great Book Report
• Mood in Composition and Literature
• How to Teach Theme
• Plot and Themes of J.R.R. Tolkien's Book 'The Hobbit'
• Anthology: Definition and Examples in Literature
• Symbols and Motifs in Literature
• 'The Devil and Tom Walker' Study Guide

University of Tasmania, Australia

Literature reviews.

• What is a literature review?
• What is a 'researchable' question?

Preliminary research

Generating ideas, refining your question, question frameworks.

• How to find the literature
• How to manage the reading and take notes that make sense
• How to bring it all together: examples, templates, links, guides

Questions vs Thesis statements

Questions and thesis statements both have the same purpose, which is to state what you are writing about and restrict your paper to a manageable topic.

• A question is phrased as a problem to solve
• A thesis statement is phrased as a  tentative answer , but both sides of the question must still be considered in your research

Badke, WB 2014,  Research strategies : finding your way through the information fog,  iUniverse, Bloomington IN.

Question - Does eating chocolate improve cognitive function?

Thesis statement - Eating chocolate may improve cognitive function.

You may find it helps to think in terms of answering a question at the beginning of your research to avoid unintentionally biasing yourself toward one answer.

Good research questions don't usually appear out of thin air.  You need to do some preliminary reading to understand the range of questions and answers relating to a topic before you can identify possible areas to focus on. 

You also need to consider the scope of your project, and make sure it is realistic given your timeframe and word count.

• Is there enough research to review? If there is not enough written on your topic to review, you will need to widen your scope, or change your question.  
• Is there too much? If there is too much literature on your topic to review, you will have to focus more tightly on one aspect.

These decisions cannot be made before you start.  Only reading about the issues that are being discussed in the existing academic literature will give you enough background knowledge to choose an achievable focus for your review.

Review articles

One way to get an overview of the body of research on your topic is to look for published review articles.

What's the difference between a primary research article and a review article?

• A primary research article is a description of a single study, investigation or experiment.
• A review article is a summary and analysis of several other studies, investigations, and experiments.

Review articles summarise the current state of understanding on a topic.

They can help you find:

• the main people working in an area of research
• recent advances and discoveries
• important primary research papers in the field
• significant gaps in the research
• current debates
• suggestions about where research might go next

IMPORTANT: please resist the temptation to use a published review article as a direct template for your own review, because this makes it very easy to unintentionally plagiarise!

Creating a mind map of topics from your preliminary reading can be a great way to generate ideas for possible research questions.  Mind mapping will allow you to:

• See an overview of your topic
• Identify connections between ideas and details
• Group emerging themes together (colour coding works well for this)

If you prefer to create a digital mind map, there are loads of software options to choose form (try Coggle , Xmind ,  Freeplane ,  MindMeister , Ayoa , or MindNode ), but I strongly urge you to consider using the traditional pen and paper method. 

Why go old-school?

Drawing your mind map by hand gives you complete freedom to concentrate on your ideas (and the relationships between them) without being distracted by trying to make software do what you want it to.

Your hand drawn mind map may not look pretty, but remember that this is a thinking tool .  It isn't supposed to be a work of art.

• Mind Mapping Worksheet (.docx)
• Mind Mapping Worksheet (.pdf)

When you have done some preliminary reading and brainstorming, you will probably find that you have several ideas that could be turned into focused research questions.  One way to do this is the tried-and-true 5W+H method:

• Refining Your Research Question Worksheet (.docx)
• Refining Your Research Question Worksheet (.pdf)

There are also several frameworks that have been developed to help researchers structure effective questions and clarify main concepts.  They are often used in Evidence Based Practice methodology for the health sciences, but some can also be useful in other disciplines.

Even when your question does not fit perfectly into any one framework, using part of one to clarify your thinking can help tremendously.

Here are some of the most common frameworks:

• PICO and variations

Useful for: clinical research

Richardson, WS, Wilson, MC, Nishikawa, J, & Hayward, RS 1995, 'The well-built clinical question: A key to evidence-based decisions',  ACP journal club, vol. 123, no. 3, pp. A12-A12.

Further resources for using PICO

Developing Your Search Question using PICO/PIO/PEO (Teeside University)

Seven steps to the perfect PICO search (EBSCO - CINAHL Complete)

Clinical examples using PICO (CIAP - Clinical Information Access Portal)

Useful for: qualitative research

Khan, KS, Kunz, R, Kleijnen, J, & Antes, G 2003,  Systematic reviews to support evidence-based medicine: How to review and apply findings of healthcare research , London: Royal Society of Medicine Press.

Useful for: qualitative / mixed methods research focused on samples

Cooke, A, Smith, D, & Booth, A 2012, 'Beyond PICO: The SPIDER tool for qualitative evidence synthesis',  Qualitative Health Research, vol. 22 no. 10, pp. 1435-1443.

Useful for: qualitative research examining a policy or service

Wildridge, V, & Bell, L 2002, 'How CLIP became ECLIPSE: A mnemonic to assist in searching for health policy/management information',  Health Information & Libraries Journal , vol. 19, no. 2, pp. 113–115.

More frameworks

A review of 38 question frameworks

Supplementary material from: Booth, A., Noyes, J., Flemming, K., Moore, G., Tunçalp, Ö., & Shakibazadeh, E. (2019). Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ global health, 4(Suppl 1), e001107. https://doi.org/10.1136/bmjgh-2018-001107

• << Previous: What is a literature review?
• Next: How to find the literature >>
• Last Updated: Apr 10, 2024 11:56 AM
• URL: https://utas.libguides.com/literaturereviews

• Reference Manager
• Simple TEXT file

People also looked at

Original research article, analyzing the relationship between consumers’ and entrepreneurs’ food waste and sustainable development using a bibliometric approach.

• 1 Department of Economics and Sustainable Development, School of Environment, Geography and Applied Economics, Harokopio University of Athens, Kallithea, Greece
• 2 Department of Public and Community Health, University of West Attica, Athens, Greece

The present study investigates the relationship between food waste and sustainable development, aiming to reveal contextual insights and present novel findings regarding the pivotal importance of waste and environmental strategies toward a circular economy. This research represents an effort to delineate methodological and thematic contributions, thoroughly analyze key themes, examine co-citation patterns, assess collaboration among countries, and identify current knowledge gaps in the literature. As waste management takes precedence within the framework of sustainable development goals, policymakers, and academia will better understand how effective food waste management can contribute to environmental sustainability. Methodologically, we employ systematic review, employing the PRISMA approach, analyzing 761 final papers, and investigating the relationship between food waste and sustainable development. We delve deeper to reveal contextual insights and present empirical findings that underscore the critical role of food waste in the economy and environment. Furthermore, guided by the identified knowledge gaps, we illuminate potential future research avenues that hold immense promise for advancing our understanding of food waste and its impact on sustainable development.

1 Introduction

Sustainable Development Goals centered on food security, environmental preservation, and optimizing material and energy usage are significant motivators for effectively managing the overuse of food waste ( Kaur et al., 2021 ). Food waste is a pressing global issue that squanders valuable resources and exacerbates challenges related to food security, environmental sustainability, and economic efficiency. Food waste, as defined by the Waste and Resources Action Programme, 1 “ is any food and inedible parts sent to a specified list of food waste destinations, where “food” is defined as any substance that was at some point intended for human consumption.”

A critical issue facing our global food system is the enormous amounts of food wasted yearly, leaving millions hungry. In order to resolve this contradiction, it is essential to comprehend the complex relationship between food waste and consumption patterns. This comprehensive investigation explores the different ways that consuming habits, from meal preparation and disposal to planning and purchasing, contribute to the creation of food waste. By looking at these relationships, we hope to pinpoint important intervention areas and create plans to encourage ethical and sustainable food consumption habits, ultimately reducing waste and guaranteeing everyone fair access to wholesome food. Approximately one-third of the food produced for human consumption goes to waste ( Schanes et al., 2018 ). This phenomenon leads to several environmental issues, such as soil erosion, deforestation, water and air pollution, and the release of greenhouse gases during various stages of food production ( Mourad, 2016 ). Thus, most developed countries have witnessed growing awareness and concern regarding the magnitude of food waste within their borders in recent years. Understanding food waste’s intricate parameters and dynamics becomes paramount as we strive to become more environmentally conscious and sustainable.

The current study broadly examines the available literature and discusses the interdisciplinary nature of food waste and its role in sustainable development ( Buczacki et al., 2021 ). Our investigation seeks to unravel the main findings of the current research on sustainable development and SDGs. In doing so, we aim to shed light on the extent of the problem, the societal, economic, and environmental repercussions, and the potential strategies and interventions that can be adopted to mitigate food waste. The world, characterized by its cultural diversity, varying consumption patterns, and dynamic economies, presents a unique and complex landscape for studying food waste. Our investigation extends academic discussion on the association between food waste and sustainability.

More specifically, this study highlights the countries, the authors, and the sources that decidedly investigate the relationship between food waste and sustainable development. By analyzing the successes and challenges faced in the region, we aim to provide valuable insights that can underline possible scientific gaps, inform policy development, and encourage cross-border collaboration in the fight against food waste. In conclusion, this research contributes to the growing knowledge surrounding food waste. By examining the shape of food waste dynamics, we hope to provide food of thought for a foundation for evidence-based policies and practices to minimize food waste’s detrimental impact on society, the environment, and the economy. Pursuing a more sustainable and food-secure future for Europe necessitates a deeper understanding of food waste, making this study an essential step toward that goal.

In sum, the main contribution of this systematic literature review is twofold. Firstly, it serves as a tool for pinpointing areas where scholarly evidence remains insufficient, highlighting the need for further research to expand our understanding of food waste behavior. Secondly, it establishes a knowledge repository that can offer valuable insights for evidence-based decision-making and policy formulation. This, in turn, can enhance the quality and efficacy of policy measures and technological innovations to reduce food waste. However, several objectives and research questions should be addressed and responded to achieve these goals. The main objectives are as follows:

• To investigate the relationship between food waste and sustainable development through a bibliometric analysis.

• To identify key themes, trends, and knowledge gaps in the existing research on food waste and its connection to sustainability within the framework of a circular economy and

• To provide valuable insights for policymakers, academics, and stakeholders working toward reducing food waste and achieving sustainable development goals.

After that, the research questions that are necessary to be addressed and be able to achieve the objectives of the current research are as follows:

• RQ1: What are the dominant themes and research trends in the literature on food waste and sustainable development within a circular economy framework, as revealed by a bibliometric analysis?

• RQ2: What key methodological approaches are employed in the existing research on food waste and sustainable development?

• RQ3: What are the prominent countries, institutions, and authors contributing to the field, and how do they collaborate on research related to food waste and sustainable development?

• RQ4: What are the critical knowledge gaps and potential future research avenues identified in the current body of literature on food waste and sustainable development?

By addressing these research questions, this study aims to offer a comprehensive and data-driven understanding of the current knowledge surrounding food waste and its connection to achieving sustainable development within a circular economy. This understanding can inform future research efforts and guide the development of effective strategies to address this pressing global challenge.

The remaining components are organized as follows: Section 2 summarizes the research methodology and data selection. Section 3 delves into the empirical findings, exploring their connections to the article’s conceptual, intellectual, and social framework. Section 4 presents the conclusion and implications for policy.

2 Scheme of the research and empirical methodology

2.1 bibliometric data.

The bibliometrics approach assesses information trends to emphasize the contributions of both individuals and research groups. We also utilize review processes to synthesize content and generate innovative policy recommendations concerning the relationship between food waste and sustainable development. For the subsequent procedures, we exclusively rely on the Scopus database, which is recognized as one of the most reliable and comprehensive sources. We used “ food waste ” and “ sustainable development ” to pinpoint publications. After that, we eliminated non-relevant publications to exclude irrelevant studies ( Shahbaz et al., 2021 ) following the PRISMA methodology ( Page et al., 2021 ).

In particular, a comprehensive search string in Scopus combined relevant keywords related to food waste and sustainable development. The exclusion criteria were as follows: duplicates were removed, non-English language articles were excluded, and conference materials, editorials, and letters to maintain focus on in-depth research were also removed. Following data extraction using a standardized form, we employed a multifaceted approach. Bibliometric software like VOSviewer and bibliometrix facilitated co-citation analysis, keyword clustering, and citation network visualization are also employed. Additionally, text analysis techniques complemented our understanding of key themes and emerging trends within the selected publications. This process narrows our selection to a final set of 761 studies from 214 sources for further examination.

A compilation of 1,480 research publications published between 2003 and 2023 is the outcome of our first search. We chose this time frame with significant consideration for the reasons listed below. First, we want to highlight some recent developments. There has been much advancement in food waste and its relationship to sustainable development in recent years. With an emphasis on studies released after 2003, we sought to encompass the most recent findings and patterns in this quickly developing subject. Second, it is critical to comprehend the most recent research findings and their implications for current policy and practice as the urgency of tackling food waste and reaching sustainable development goals increases.

2.2 Bibliometric analysis

Researchers can use quantitative and qualitative methodologies to identify gaps in the scientific literature by using the bibliometrics methodology to track trends in academic research ( Siddiqui et al., 2023 ). We have used bibliometric tools like VOSviewer, the R-package, and Biblioshiny to analyze publications about food waste ( Aria and Cuccurullo, 2017 ). The VOSviewer is a tool that uses a two-dimensional map to show the relationships between co-citation data, geographic locations, research journals, and keywords. Their proximity shows the degree of link or similarity between nodes in this visualization. More specifically, this software is excellent at producing two-dimensional maps showing the connections between various items in a dataset. The most popular keywords and how they gathered together are shown in the visualization, which sheds light on the recurring themes in food waste. In addition, VOSviewer assists us in recognizing significant research and schools of thinking that have shaped our current comprehension of food waste and its relationship to sustainability.

On the other hand, R-package and Biblioshiny utilize a diverse set of bibliometric tools that serve as visualization functions for conducting information analysis and generating scientific maps related to the intersection of food waste and sustainability ( da Silva Duarte et al., 2021 ; Srinivas, 2022 ). The package facilitates efficient data processing and transformation, ensuring the accuracy and consistency of the analysis. Biblioshiny provides advanced functions for constructing and analyzing bibliographic networks, allowing us to explore the intricate relationships between different entities within the food waste literature.

3 Empirical results

3.1 publication output and citation growth.

Figure 1 presents a per annum publication and citation growth trend since 2003, with an average of 75.6 citations per document. Notably, interdisciplinary research on food waste has received significant attention recently ( Dhir et al., 2020 ). There has been an exponential increase in publications in recent years, with 2023 having the highest number of publications (146 articles).

Figure 1 . Publication output.

Document-citation analysis was also performed using the “document” unit from the downloaded publications to create a table based on citation data by selecting the first ten documents as a threshold. For each of the ten documents, the number of citations and doi number are presented in the following Table 1 . For brevity, most citations reported by Papargyropoulou et al. (2014) (910 citations) have investigated the factors that increase food waste through several channels of the food supply chain and propose a framework for appropriately managing food waste. Guo et al. (2010) (636 citations) focused their interest on agricultural production and the degradation of the natural environment due to the energy crisis. Authors propose hydrogen as one of the most promising substitutes for fossil fuels. After that, a group of authors with around 300 citations consists of Notarnicola et al. (2017) , who has a total of 383 citations; Williams et al. (2012) , with 360 citations, and Xue et al. (2017) , who has 359 citations. Next, Alexander et al. (2017) , Mourad, Garrone et al. (2014) , Sharma P. et al. (2020) , and Sharma S. et al. (2020) papers have more than 200 citations, but less than 300 completed the first ten high-cited documents.

Table 1 . Top 10 most cited documents.

3.2 Countries’ collaboration networks

Next, Figures 2 , 3 visually represent global research collaboration among countries, displaying the collaboration network and the volume of publications contributed by each country.

Figure 2 . Country scientific production.

Figure 3 . Countries’ collaboration network.

In Figure 2 , the research output is presented with varying shades of color, wherein the darker colors represent the regions with the highest frequency of publications. Notably, China is the global leader in research publications, with an impressive count of 481, showcasing its substantial contribution to the academic landscape. Following closely behind are other key players in the research arena, with Italy contributing 442 publications, the United Kingdom with 246, India with 215, and the United States with 208, all demonstrating their significant presence in the global research community. Additionally, several other highly productive economies, such as Spain, Sweden, Germany, Malaysia, and Australia, are notable contributors, further enriching the global research output landscape.

Moving on to Figure 3 , it describes the collaborative aspect of research on a global scale, shedding light on the interconnections and partnerships between various countries in the pursuit of knowledge and academic advancement.

A compelling pattern emerges when examining collaborations between authors and countries in food waste. Notably, China, Italy, and the United Kingdom substantially collaborate. This outcome highlights their proactive stance in fostering international partnerships to tackle food waste and sustainability nexus. Following closely behind are the United States and Spain, both of which also participate actively in collaborative initiatives. These findings underscore the global significance of addressing food waste and the willingness of these nations to join forces in addressing this critical challenge.

3.3 Keywords, authors, and key countries framework

The upcoming section aims to reveal how researchers have documented various research streams across different countries. To achieve this, we employ the CAK framework to introduce innovative visualizations that portray the amalgamation of authors, research themes, and countries.

It is evident from Figure 4 that Italy, China, and the UK are the most prominent geographical locations. Likewise, the most dominant research themes are food waste, sustainable development, waste management, and waste disposal. It is worth noting that the smaller size of countries with limited contributions suggests that the current state of research is in its early stages. Additional research, mainly from European economies with substantial food demands, should shed light on recent research developments and explore new avenues of inquiry.

Figure 4 . CAK framework. Authors, keywords, countries.

Figure 5 , depicted as a tree diagram, visually represents the keywords extensively employed in the array of previously studied records. A closer examination of the results illuminates the prominent themes authors have chosen to emphasize in their works.

Figure 5 . Keywords tree-map.

Notably, “food waste” takes the lead, featuring in approximately 36% of the articles. This underscores the paramount significance of addressing food waste within the scope of the research, signifying its pervasive relevance in current academic discourse. Sustainability is another crucial focus in 18% of the articles, highlighting the shared commitment to promoting sustainable practices and environmental responsibility within food waste management. Furthermore, the concept of a “circular economy” garners notable attention, being employed as a keyword in 10% of the publications, reflecting the growing interest in developing circular and resource-efficient systems to combat food waste. The utilization of “life cycle assessment” as a keyword in 8% of the articles underscores the methodological approach many authors took, emphasizing the importance of assessing environmental impacts across the entire life cycle of food products. Finally, the notion of “sustainable development” is reflected in 7% of the works, signifying the broader context in which food waste mitigation is situated, emphasizing the pursuit of development that satisfies current requirements without jeopardizing those of coming generations. This breakdown of prevalent keywords offers valuable insights into the thematic and methodological orientations of the scholarly discourse surrounding food waste. It underscores the critical areas of focus within this research domain.

3.4 Co-citation analysis of authors – intellectual structure

Afterward, we utilize co-citation analysis to understand better how literature has evolved in recent decades. The extent to which studies reference one another indicates the interrelatedness within the scientific literature. Co-citation analysis is a constantly changing metric that aids in recognizing emerging paradigms within a selected body of academic literature ( Buczacki et al., 2021 ). In our present study, we refer to Figure 6 (co-citation analysis), where the number of citations is represented, and the relatedness of topics is indicated by the distance between these nodes, shedding light on academic discourse. The visualization in Figure 6 reveals two distinct clusters.

Figure 6 . Co-citations analysis of authors.

The first group of publications delves into the importance of food waste as a critical element in developing a sustainable food system ( Quested et al., 2011 ). They also attempt to identify the losses occurring along the entire food chain and identify the causes of food losses and possible ways of preventing them ( Quested et al., 2011 ; Falasconi et al., 2015 ; Eriksson et al., 2020 ). Conversely, a second cluster examines food issues in a more global scale analysis linked to sustainable development goals ( Liu et al., 2022 ) introducing, for instance, the effects of international food trade on the food system ( Wang et al., 2022 ). Smaller groups use a more quantitative analysis highlighting possible environmental and socio-economic impacts of food waste ( Albizzati et al., 2021 ) and policies at a micro level ( Lassen et al., 2019 ).

3.5 Conceptual structure of the publications

Recently, keyword co-occurrence networks have become increasingly popular in systematic review-based studies, offering a means to harness knowledge mapping and uncover associations among research themes in research management ( Bashir et al., 2021 ). This approach empowers researchers to comprehensively understand a specific field within the amassed knowledge, harnessing the associations between keywords to reveal insights in economic literature. In our current research, we utilize a keyword co-occurrence network approach, setting a threshold of at least five occurrences for a word to be included. Consequently, out of a total of 1,136 keywords, 113 satisfied this requirement (see Figures 7 , 8 ).

Figure 7 . Keywords co-occurrence network.

Figure 8 . Keywords co-occurrence network.

Notably, keywords like “food waste,” “sustainability,” “waste management,” “environmental impact,” and “waste disposable” are the most frequently occurring. Furthermore, as depicted in Figure 7 , three separate groups of keywords are evident (green, red, and blue). Looking at the blue cluster, the keywords “sustainability,” “food supply,” “food security,” “nutrition” and “supply chain” exhibit close associations. As far as the red cluster is concerned, the keywords “article,” “waste disposal,” “fertilizer,” “biogas,” and “nitrogen” indicate a group of research that investigates the concept of food waste from a different perspective. An interesting observation is that keywords (green cluster) such as “life cycle assessment,” “gas emissions,” “anaerobic digestion,” “municipal waste,” and “climate change” have small node sizes but remain interconnected in terms of links.

After that, we expanded the keywords’ co-occurrence network by exploring its time evolution. The connection between food waste and sustainability is relatively recent, with most publications emerging after 2019. Even more recently, authors have also incorporated into their analysis the “circular economy,” “sustainable development goals,” “waste management,” and “food supply.” These keywords provide more information on the trend already in process around the relationship between food waste management and sustainability in the future. A notable trend is observed, with most publications centered around food waste, sustainability, environmental impact assessment, food supply, and climate change.

We further employ Figure 9 to explore thematic mapping from the perspective of four distinct subdivisions, which aids in comprehending the diversity and significance of sub-components within the scientific literature ( Buczacki et al., 2021 ). We have established the maximum number of keywords at 119 and the minimum cluster frequency at 3.

Figure 9 . Thematic map.

The upper right section encompasses “motor themes,” representing research topics with the highest density and centrality, such as “food waste,” “sustainability,” “circular economy,” “life cycle assessment,” and “waste management.” In the lower-right quadrant, we find “food security” characterized by low density, discussing topics like “recycling,” “climate change,” “food waste management,” and “food supply chain.” These transversal themes hold significant importance in the research, contributing to discussions on various research directions. Lastly, “declining or emerging themes” and “niche themes” encompass research topics related to “household food waste” and “co-digestion.” In summary, Figure 9 serves as a valuable tool for understanding the current academic discourse and the potential role of food waste in future waste management policies.

4 Discussion

The present work aims to offer researchers and policymakers a toolbox of organized ideas to tackle food waste. Simultaneously, to achieve the objectives outlined in the Sustainable Development Goals, the target of reducing food waste and adopting a comprehensive strategy incorporating various measures is domineering. These efforts rectify informational gaps already highlighted by previous systematic literature review works ( Schanes et al., 2018 ).

Besides, similar to previous studies ( Principato et al., 2021 ) our research sheds light on the complex sides of the food waste phenomenon, highlighting the trend of research on this topic ( Zhang et al., 2018 ; Vásquez Neyra et al., 2022 ; D'Adamo et al., 2023 ). More specifically, our study provides an essential segment of information presenting the issues around food waste that are denoted as the motor themes with a critical role of food waste in future food waste management and policies, such as the concept of circular economy ( de Oliveira et al., 2021 ; Nikolaou and Tsagarakis, 2021 ; Santagata et al., 2021 ). In this direction, the European Food Safety Authority (2020) highlights that circular economy initiatives are increasing attention to food waste as a food and feed source.

Furthermore, our review, concentrating on strategies to reduce food waste and promote sustainability, dovetails with the Green Deal’s priorities, such as resource efficiency, waste reduction, and sustainable consumption. By positioning the study within the broader context of global sustainability challenges, it becomes evident that addressing food waste is integral to achieving the Green Deal’s ambitious targets. In this direction, according to FAO (2021) one of the ways toward more sustainable agriculture and food production is to manage food production systems sustainably through significant reductions in food loss and waste.

5 Concluding remarks and policy implications

The central objective of our research is to investigate the relationship between food waste and sustainability through a review of academic literature. Our study thoroughly examines all pertinent publications on the nexus between food waste and sustainable development. Given the pivotal role of food waste in the context of Sustainable Development Goals (SDGs), it becomes crucial for policymaking institutions to evaluate socio-economic and policy variables.

This evaluation is essential for harmonizing food consumption and environmental sustainability via several policy implications. For instance, policymakers should prioritize integrating food waste reduction strategies into the broader framework of SDGs. This approach ensures a more holistic and sustainable approach to addressing food waste while advancing the global sustainability agenda. Also, at a microeconomic level, governments must focus on socio-economic and policy factors that directly influence food waste. By designing policies that incentivize food waste reduction at the individual, household, and industrial levels, they can contribute to achieving both economic and environmental goals. On a more global-scale and macroeconomic level, collaboration among countries, mainly focusing on emerging economies, is essential to address the multifaceted challenges posed by food waste. Policymakers should explore international partnerships to facilitate knowledge sharing, best practices, and innovative strategies in mitigating food waste.

In a more specific and focused aspect, our analysis identifies key themes and research areas related to food waste and sustainability. European and national policy measures should emphasize the need to integrate specific food waste reduction strategies into national SDG goals such as Goal 2 (zero hunger) and Goal 12 (responsible consumption and production). Moreover, governments and policymakers can identify specific socio-economic and policy factors that significantly influence food waste at individual, household, and industrial levels. These policy interventions could include consumer awareness campaigns, incentivizing food waste reduction within entrepreneurs, exploring policies like tax breaks for businesses implementing waste reduction strategies, or introducing waste disposal fees based on waste generation. Financially, the public sector could also encourage policies that facilitate the redistribution of surplus food to those in need, reducing waste and promoting social welfare. That can be done by promoting the establishment of international funding mechanisms to support emerging economies in implementing effective food waste reduction strategies.

However, it is noteworthy that conducting country-specific analyses can offer valuable insights and potentially address limitations associated with quantitative data and analysis. Understanding the specific dynamics of food waste in different nations is crucial for tailoring effective policies and interventions. In addition, we urge future research to delve into the role of addressing food waste issues, particularly in emerging economies. Such research endeavors have the potential to yield diverse policy insights. By examining the unique challenges and opportunities food waste presents in these regions, policymakers can better understand the evolving issues in developing and developed nations. This knowledge is invaluable for shaping effective strategies to reduce food waste and promote sustainable practices worldwide.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

IK: Conceptualization, Investigation, Software, Visualization, Supervision, Visualization, Writing – original draft, Writing – review & editing. SP: Investigation, Software, Visualization, Writing – original draft, Writing – review & editing. GM: Investigation, Methodology, Software, Visualization, Writing – original draft, Writing – review & editing.

The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

1. ^ https://wrap.org.uk/

Albizzati, P. F., Tonini, D., and Astrup, T. F. (2021). High-value products from food waste: an environmental and socio-economic assessment. Sci. Total Environ. 755:142466. doi: 10.1016/j.scitotenv.2020.142466

PubMed Abstract | Crossref Full Text | Google Scholar

Alexander, P., Brown, C., Arneth, A., Finnigan, J., Moran, D., and Rounsevell, M. D. (2017). Losses, inefficiencies and waste in the global food system. Agric. Syst. 153, 190–200. doi: 10.1016/j.agsy.2017.01.014

Aria, M., and Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics , 11, 959–975. doi: 10.1016/j.joi.2017.08.007

Crossref Full Text | Google Scholar

Bashir, M. F., Ma, B., Qin, Y., and Bashir, M. A. (2021). Evaluation of one belt one road publications: a bibliometric and literature review analysis. Environ. Sci. Pollut. Res. 28, 37016–37030. doi: 10.1007/s11356-021-14621-y

Buczacki, A., Gładysz, B., and Palmer, E. (2021). HoReCa food waste and sustainable development goals—a systemic view. Sustain. For. 13:5510. doi: 10.3390/su13105510

D'Adamo, I., Desideri, S., Gastaldi, M., and Tsagarakis, K. P. (2023). Sustainable food waste management in supermarkets. Sustain. Prod. Consum. 43, 204–216. doi: 10.1016/j.spc.2023.11.005

de Oliveira, M. M., Lago, A., and Dal’Magro, G. P. (2021). Food loss and waste in the context of the circular economy: a systematic review. J. Clean. Prod. 294:126284. doi: 10.1016/j.jclepro.2021.126284

da Silva Duarte, K. S., Lima, T. A. C., Alves, L. R., Rios, P. A. P., and Motta, W. H. (2021). The circular economy approach for reducing food waste: a systematic review. Revista Produção e desenvolvimento 7. doi: 10.32358/rpd.2021.v7.572

Dhir, A., Talwar, S., Kaur, P., and Malibari, A. (2020). Food waste in hospitality and food services: A systematic literature review and framework development approach. Journal of Cleaner Production , 270, 122861. doi: 10.1016/j.jclepro.2020.122861

Eriksson, M., Malefors, C., Bergström, P., Eriksson, E., and Persson Osowski, C. (2020). Quantities and quantification methodologies of food waste in Swedish hospitals. Sustain. For. 12:3116. doi: 10.3390/su12083116

European Food Safety AuthorityMaggiore, A., Afonso, A., Barrucci, F., and Sanctis, G. D. (2020). Climate change as a driver of emerging risks for food and feed safety, plant, animal health and nutritional quality. EFSA Support. Publ. 17:1881E. doi: 10.2903/sp.efsa.2020.EN-1881

Falasconi, L., Vittuari, M., Politano, A., and Segrè, A. (2015). Food waste in school catering: an Italian case study. Sustain. For. 7, 14745–14760. doi: 10.3390/su71114745

FAO. (2021) In Brief to The State of Food and Agriculture “ Making agrifood systems more resilient to shocks and stresses .” Rome, FAO.

Google Scholar

Garrone, P., Melacini, M., and Perego, A. (2014). Opening the black box of food waste reduction. Food Policy 46, 129–139. doi: 10.1016/j.foodpol.2014.03.014

Guo, X. M., Trably, E., Latrille, E., Carrère, H., and Steyer, J. (2010). Hydrogen production from agricultural waste by dark fermentation: a review. Int. J. Hydrog. Energy 35, 10660–10673. doi: 10.1016/j.ijhydene.2010.03.008

Kaur, G., Lasaridi, K., and Wong, J. (2021). “Sustainable food waste management: an introduction” in Current developments in biotechnology and bioengineering (Elsevier), 1–10.

Lassen, A. D., Christensen, L. M., Spooner, M. P., and Trolle, E. (2019). Characteristics of canteens at elementary schools, upper secondary schools and workplaces that comply with food service guidelines and have a greater focus on food waste. Int. J. Environ. Res. Public Health 16:1115. doi: 10.3390/ijerph16071115

Liu, Y., Tan, Q., Chen, J., Pan, T., Penuelas, J., Zhang, J., et al. (2022). Dietary transition determining the tradeoff between global food security and sustainable development goals varied in regions. Earth's Future 10:e2021EF002354 doi: 10.1029/2021EF002354

Mourad, M. (2016). Recycling, recovering and preventing “food waste”: competing solutions for food systems sustainability in the United States and France. J. Clean. Prod. 126, 461–477. doi: 10.1016/j.jclepro.2016.03.084

Nikolaou, I. E., and Tsagarakis, K. P. (2021). An introduction to circular economy and sustainability: some existing lessons and future directions. Sustain. Prod. Consum. 28, 600–609. doi: 10.1016/j.spc.2021.06.017

Notarnicola, B., Sala, S., Anton, A., McLaren, S. J., Saouter, E., and Sonesson, U. (2017). The role of life cycle assessment in supporting sustainable Agri-food systems: a review of the challenges. J. Clean. Prod. 140, 399–409. doi: 10.1016/j.jclepro.2016.06.071

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., et al. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Journal of Clinical Epidemiology , 134, 178–89. https://www.ncbi.nlm.nih.gov/pubmed/33789819

Papargyropoulou, E., Lozano, R., Steinberger, J. K., Wright, N., and bin Ujang, Z. (2014). The food waste hierarchy as a framework for the management of food surplus and food waste. J. Clean. Prod. 76, 106–115. doi: 10.1016/j.jclepro.2014.04.020

Principato, L., Mattia, G., Di Leo, A., and Pratesi, C. A. (2021). The household wasteful behaviour framework: a systematic review of consumer food waste. Ind. Mark. Manag. 93, 641–649. doi: 10.1016/j.indmarman.2020.07.010

Quested, T. E., Parry, A. D., Easteal, S., and Swannell, R. (2011). Food and drink waste from households in the UK , vol. 36, 460–467.

Santagata, R., Ripa, M., Genovese, A., and Ulgiati, S. (2021). Food waste recovery pathways: challenges and opportunities for an emerging bio-based circular economy. A systematic review and an assessment. J. Clean. Prod. 286:125490 doi: 10.1016/j.jclepro.2020.125490

Schanes, K., Dobernig, K., and Gözet, B. (2018). Food waste matters-a systematic review of household food waste practices and their policy implications. J. Clean. Prod. 182, 978–991. doi: 10.1016/j.jclepro.2018.02.030

Siddiqui, A., Altekar, S., Kautish, P., Fulzele, S., Kulkarni, N., Siddiqui, M., et al. (2023). Review of measurement of sustainable development goals: A comprehensive bibliometric and visualized analysis. Environmental Science and Pollution Research , 30, 91761–91779. doi: 10.1007/s11356-023-28887-x

Shahbaz, M., Bashir, M. F., Bashir, M. A., and Shahzad, L. (2021). A bibliometric analysis and systematic literature review of tourism-environmental degradation nexus. Environ. Sci. Pollut. Res. 28, 58241–58257. doi: 10.1007/s11356-021-14798-2

Sharma, S., Basu, S., Shetti, N. P., and Aminabhavi, T. M. (2020). Waste-to-energy nexus for circular economy and environmental protection: recent trends in hydrogen energy. Sci. Total Environ. 713:136633. doi: 10.1016/j.scitotenv.2020.136633

Sharma, P., Gaur, V. K., Kim, S., and Pandey, A. (2020). Microbial strategies for bio-transforming food waste into resources. Bioresour. Technol. 299:122580. doi: 10.1016/j.biortech.2019.122580

Srinivas, H. (2022). “Nexus in green economics for Asian countries” in Progress in green economics , 1–16.

Vásquez Neyra, J. M., Cequea, M. M., Gomes Haensel Schmitt, V., and Ferasso, M. (2022). Food consumption and food waste behaviour in households in the context of the COVID-19 pandemic. Br. Food J. 124, 4477–4495. doi: 10.1108/BFJ-07-2021-0798

Wang, J. M., Liu, Q., Hou, Y., Qin, W., Bai, Z. H., Zhang, F. S., et al. (2022). Impacts of international food and feed trade on nitrogen balances and nitrogen use efficiencies of food systems. Sci. Total Environ. 838:156151. doi: 10.1016/j.scitotenv.2022.156151

Williams, H., Wikström, F., Otterbring, T., Löfgren, M., and Gustafsson, A. (2012). Reasons for household food waste with special attention to packaging. J. Clean. Prod. 24, 141–148. doi: 10.1016/j.jclepro.2011.11.044

Xue, L., Liu, G., Parfitt, J., Liu, X., Van Herpen, E., Stenmarck, Å., et al. (2017). Missing food, missing data? A critical review of global food losses and food waste data. Environ. Sci. Technol. 51, 6618–6633. doi: 10.1021/acs.est.7b00401

Zhang, M., Gao, M., Yue, S., Zheng, T., Gao, Z., Ma, X., et al. (2018). Global trends and future prospects of food waste research: a bibliometric analysis. Environ. Sci. Pollut. Res. 25, 24600–24610. doi: 10.1007/s11356-018-2598-6

Keywords: food waste, sustainability, behavior, bibliometric analysis, VOSviewer, bibliometrix

Citation: Kostakis I, Papadaki S and Malindretos G (2024) Analyzing the relationship between consumers’ and entrepreneurs’ food waste and sustainable development using a bibliometric approach. Front. Sustain . 5:1373802. doi: 10.3389/frsus.2024.1373802

Received: 20 January 2024; Accepted: 15 March 2024; Published: 04 April 2024.

Reviewed by:

Copyright © 2024 Kostakis, Papadaki and Malindretos. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Ioannis Kostakis, [email protected]

This article is part of the Research Topic

Innovation in Sustainable Food

This paper is in the following e-collection/theme issue:

Published on 10.4.2024 in Vol 26 (2024)

Methodological Frameworks and Dimensions to Be Considered in Digital Health Technology Assessment: Scoping Review and Thematic Analysis

Authors of this article:

• Joan Segur-Ferrer, BSS, PT, MSc   ; 
• Carolina Moltó-Puigmartí, BScPharm, PhD   ; 
• Roland Pastells-Peiró, BA, MA, MsC   ; 
• Rosa Maria Vivanco-Hidalgo, MD, MPH, PhD  

Agency for Health Quality and Assessment of Catalonia, Barcelona, Spain

Corresponding Author:

Joan Segur-Ferrer, BSS, PT, MSc

Agency for Health Quality and Assessment of Catalonia

Roc Boronat Street, 81-95, 2nd Fl

Barcelona, 08005

Phone: 34 935 513 900

Fax:34 935 517 510

Email: [email protected]

Background: Digital health technologies (dHTs) offer a unique opportunity to address some of the major challenges facing health care systems worldwide. However, the implementation of dHTs raises some concerns, such as the limited understanding of their real impact on health systems and people’s well-being or the potential risks derived from their use. In this context, health technology assessment (HTA) is 1 of the main tools that health systems can use to appraise evidence and determine the value of a given dHT. Nevertheless, due to the nature of dHTs, experts highlight the need to reconsider the frameworks used in traditional HTA.

Objective: This scoping review (ScR) aimed to identify the methodological frameworks used worldwide for digital health technology assessment (dHTA); determine what domains are being considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Methods: The ScR was performed in accordance with the guidelines established in the PRISMA-ScR guidelines. We searched 7 databases for peer reviews and gray literature published between January 2011 and December 2021. The retrieved studies were screened using Rayyan in a single-blind manner by 2 independent authors, and data were extracted using ATLAS.ti software. The same software was used for thematic analysis.

Results: The systematic search retrieved 3061 studies (n=2238, 73.1%, unique), of which 26 (0.8%) studies were included. From these, we identified 102 methodological frameworks designed for dHTA. These frameworks revealed great heterogeneity between them due to their different structures, approaches, and items to be considered in dHTA. In addition, we identified different wording used to refer to similar concepts. Through thematic analysis, we reduced this heterogeneity. In the first phase of the analysis, 176 provisional codes related to different assessment items emerged. In the second phase, these codes were clustered into 86 descriptive themes, which, in turn, were grouped in the third phase into 61 analytical themes and organized through a vertical hierarchy of 3 levels: level 1 formed by 13 domains, level 2 formed by 38 dimensions, and level 3 formed by 11 subdimensions. From these 61 analytical themes, we developed a proposal for a methodological framework for dHTA.

Conclusions: There is a need to adapt the existing frameworks used for dHTA or create new ones to more comprehensively assess different kinds of dHTs. Through this ScR, we identified 26 studies including 102 methodological frameworks and tools for dHTA. The thematic analysis of those 26 studies led to the definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered in dHTA.

Introduction

Digital health technologies (dHTs) are driving the transformation of health care systems. They are changing the way in which health services are delivered, and showing great potential to address some of the major challenges that European health systems, including the Spanish National Health System (SNS), are facing, such as the progressive aging of the population [ 1 , 2 ]; the growing demand for health and long-term care services [ 2 ]; the rise in health care costs, increasing financial pressures on health and welfare systems [ 1 , 3 ]; and the unequal distribution of health services across different geographical regions [ 4 , 5 ]. In addition, dHT can improve the accessibility, sustainability, efficiency, and quality of health care systems [ 6 , 7 ], leading to their becoming a determinant of health on their own [ 6 , 8 ].

However, the digital transformation of health care systems and the implementation of dHT (eg, artificial intelligence [AI]–based solutions, data-driven health care services, or the internet of things) are slow and unequal across different European regions [ 9 , 10 ]. Some of the reasons for this are (1) the immaturity of regulatory frameworks for the use of dHTs [ 9 ], (2) the lack of funding and investment for the implementation of dHTs [ 9 ], (3) the lack of sufficient and appropriate infrastructures and common standards for data management [ 6 , 9 ], (4) the absence of skills and expertise of professionals and users [ 10 ], and (5) the scarcity of strong evidence regarding the real benefits and effects of dHTs on health systems and people’s well-being, as well as the cost-effectiveness of these technologies. This makes decision-making difficult, potentially leading to the development and reproduction of low-value and short-lived dHTs [ 6 , 11 ].

To overcome these challenges, harness the potential of dHTs, and avoid nonintended consequences, the World Health Organization (WHO) [ 4 , 11 ] states that dHTs should be developed under the principles of transparency, accessibility, scalability, privacy, security, and confidentiality. Their implementation should be led by robust strategies that bring together leadership, financial, organizational, human, and technological resources, and decisions should be guided by the best-available evidence [ 4 , 11 ].

Regarding this last aspect, health technology assessment (HTA), defined as a “multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its life cycle,” is a widely accepted tool to inform decision-making and promote equitable, efficient, and high-quality health systems [ 12 , 13 ].

Generally, HTA is conducted according to specific methodological frameworks, such as the HTA Core Model of the European Network for Health Technology Assessment (EUnetHTA) [ 14 ] and the guidelines for the development and adaptation of rapid HTA reports of the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS) [ 15 ]. These frameworks establish the methodologies to follow and the elements to evaluate. Although these frameworks are helpful instruments for evaluating various health technologies, they have certain limitations in comprehensively assessing dHTs. For this reason, in the past few years, different initiatives have emerged to adapt existing methodological frameworks or develop new ones. The objective is to consider additional domains (eg, interoperability, scalability) to cover the intrinsic characteristics of dHTs [ 16 - 18 ]. Examples of these initiatives are the Evidence Standard Framework (ESF) of National Institute for Health and Care Excellence (NICE) [ 19 ] or the Digi-HTA Framework of the Finnish Coordinating Center for Health Technology Assessment (FinCCHTA) [ 16 ]. Nonetheless, the majority of these frameworks have certain constraints, such as being designed for a particular socioeconomic or national setting, which restricts their transferability or suitability for use in other countries; the specificity or exclusion of certain dHTs, resulting in limitations in their application; or the limited evidence regarding their actual usefulness.

In this context, we performed a scoping review (ScR) with the aim of identifying the methodological frameworks that are used worldwide for the evaluation of dHTs; determining what dimensions and aspects are considered for each type of dHT; and generating, through a thematic analysis, a proposal for a methodological framework that is based on the most frequently described dimensions in the literature. This research focused mainly on mobile health (mHealth), non–face-to-face care models and medical devices that integrate AI, as these particular dHTs are the ones most frequently assessed by HTA agencies and units of RedETS.

Identifying Research Questions

This ScR followed by a thematic analysis answered the following research questions:

• What methodological frameworks currently exist for digital health technology assessment (dHTA)?
• What domains and dimensions are considered in dHTA?
• Do the different domains and dimensions considered depend on whether the dHT addressed is a non–face-to-face care model of health care provision, a mobile device (mHealth), or a device that incorporates AI?

Overview of Methods for Conducting the Scoping Review

We conducted an ScR of the literature and a thematic analysis of the studies included according to the published protocol [ 20 ]. The ScR aimed to answer the first research question, while the thematic analysis aimed to answer the second and third research questions. Spanish experts from various domains of HTA and dHT collaborated throughout the study design and development.

The ScR of the available scientific literature was carried out in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews) guidelines ( Multimedia Appendix 1 ) [ 21 ] and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ].

Ethical Considerations

As this work was an ScR, no ethical board approval was required.

Search Strategy

The search strategy ( Multimedia Appendix 2 ) was designed by an experienced information specialist (author RP-P) in accordance with the research questions and using the validated filter of Ayiku et al [ 24 ] for health apps, adding the terms for concepts related to mHealth, remote care models, AI, digital health, methodological frameworks, and HTA. The strategy was peer-reviewed according to the “Peer Review of Electronic Search Strategies Statement” [ 25 ] by authors JS-F and CM-P and was executed in the following 7 databases, considering the characteristics of each in terms of syntax, controlled vocabulary, and proximity operators: Medline (OVID), CINAHL Plus, Embase, Cochrane Library, Scopus, Web of Science, and TripDatabase. Note that no time, language, or other filters were used.

The identification of relevant studies was complemented with a manual search based on the references in the included studies, as well as the websites of the HTA agencies identified through the web pages of EUnetHTA, the International Network for Agencies for Health Technology Assessment (INAHTA), and Health Technology Assessment International (HTAi). Additionally, a search was conducted in Google Scholar, limiting the results to the first 250 items in order to guarantee the inclusion of all pertinent studies [ 26 ].

Inclusion and Exclusion Criteria

The inclusion criteria used in the reference-screening process were based on the previously detailed research questions and are outlined in Textbox 1 using the Population/Problem, Phenomenon of Interest, Context and Design (PICo-D) format [ 27 , 28 ]. The PICo-D format was used instead of the traditional Population/Problem, Intervention, Comparator, Outcomes, Design (PICO-D) format due to the qualitative nature of the research questions and the characteristics of the phenomenon of interest.

Studies were excluded if they were published before 2011, due to the rapid evolution of dHTs in the past few years, did not describe dimensions or evaluation criteria, or were based on methodological frameworks not intended for the assessment of dHTs (eg, EUnetHTA Core Model 3.0). Likewise, we excluded comments, editorials, letters, conference abstracts, frameworks, or tools focusing on the evaluation of dHTs by users (eg, User version of Mobile App Rating Scale [uMARS]) or documents in languages other than English, Spanish. or Catalan.

Population/problem

Digital health technology assessment (dHTA)

Phenomenon of interest

Specific methodological frameworks for the evaluation of digital health (with special focus on mobile health [mHealth]: non–face-to-face care models and medical devices that integrate artificial intelligence [AI] due the type of technologies mostly assessed in the Spanish National Health System [SNS]) that describe the domains to be evaluated in dHTA

Health technology assessment (HTA)

Methodological guidelines and frameworks, scoping reviews (ScRs), systematic reviews (SRs), consensus documents, and qualitative studies

Reference Screening and Data Extraction

The screening of studies was carried out by authors CM-P and JS-F in 2 phases in accordance with the selection criteria detailed earlier ( Textbox 1 ) and in a single-blind peer review manner. The first phase consisted of screening of the titles and abstracts of the studies identified in the bibliographic search. The second phase consisted of full-text screening of the studies included in the previous phase.

Data extraction was performed by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 (Scientific Software Development GmbH) [ 29 ] and the data extraction sheets designed ad hoc for this purpose following the recommendations of the Cochrane Handbook for Systematic Reviews of Interventions [ 30 ].

When disagreements emerged in either of the 2 processes, a consensus was reached between the 3 reviewers (CM-P, RP-P, and JS-F). When a consensus was not possible, a fourth reviewer (author RMV-H) was consulted.

Collecting, Summarizing, and Reporting the Results

A descriptive analysis was carried out to evaluate and report the existing methodological frameworks and their characteristics.

Overview of Methods for Thematic Analysis

The thematic analysis was performed following the recommendations and phases described by Thomas and Harden [ 31 ] to determine HTA dimensions for dHTs: (1) line-by-line text coding, (2) development of descriptive topics, and (3) generation of analytical themes. Both analyses were carried out by 3 authors (CM-P, RP-P, and JS-F) using the web and desktop versions of ATLAS.ti version 22.0 [ 29 ].

Dimensions identified from systematic reviews (SRs) that were derived from primary studies also identified in our systematic search were only counted once in order to avoid duplication of data and risk of bias. It is worth mentioning that the primary studies included in the SRs were not directly analyzed but were analyzed through the findings reported in the SRs.

Study Selection and Characteristics

A total of 3042 studies were retrieved throughout the systematic (n=3023, 99.4%) and the manual (n=19, 0.6%) search. Of these, 2238 (73.6%) studies were identified as unique after removing duplicates.

After title and abstract review, 81 (3.6%) studies were selected for full-text review, of which 26 (32.1%) were finally included in the analysis. The excluded studies and reasons for exclusion are detailed in Multimedia Appendix 3 ; in brief, the reasons for exclusion were phenomenon of interest (n=30, 37%), type of publication (n=15, 18.5%), purpose (n=6, 7.4%), language (n=2, 2.5%), and duplicated information (n=2, 2.5%). The study selection process is outlined in Figure 1 [ 32 ].

Of the 26 (32.1%) studies included in this ScR, 19 (73.1%) were designed as specific methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ], 4 (15.4%) were SRs [ 48 - 51 ], 1 (3.9%) was a report from the European mHealth Hub’s working group on mHealth assessment guidelines [ 52 ], 1 (3.9%) was a qualitative study [ 53 ], and 1 (3.9%) was a viewpoint [ 54 ]. In addition, 3 (11.5%) focused on the assessment of non–face-to-face care models [ 33 - 35 ], 8 (30.8%) on mHealth assessment [ 36 - 40 , 52 , 53 , 55 ], 2 (7.7%) on the assessment of AI technology [ 41 , 54 ], 4 (15.4%) on eHealth [ 42 , 43 , 48 , 50 ], and 9 (34.6%) on the overall assessment of digital health [ 16 , 17 , 44 - 47 , 49 , 51 , 56 ].

Research Question 1: Description of Identified Frameworks for dHTA

The 19 methodological frameworks for dHTA [ 16 , 17 , 33 - 47 ] were from various countries: The majority (n=5, 26.3%) originated in Australia [ 17 , 34 , 38 , 41 , 46 ], followed by 3 (15.8%) from the United States [ 43 , 45 , 56 ] and 2 (10.5%) from Switzerland [ 47 , 55 ]; the remaining 9 (47.4%) frameworks were developed in Afghanistan [ 42 ], Denmark [ 33 ], Scotland [ 35 ], Finland [ 16 ], Ireland [ 36 ], Israel [ 40 ], the United Kingdom [ 37 ], Spain [ 39 ], and Sweden [ 44 ].

The 19 methodological frameworks focused on evaluating various types of technologies. Specifically, 3 (15.8%) of them were designed for assessing non–face-to-face care models [ 33 - 35 ], 6 (31.6%) for mHealth [ 36 - 40 ], and 1 (5.3%) for AI solutions [ 41 ]. The other 9 (47.4%) frameworks addressed eHealth [ 42 , 43 , 56 ] or digital health in general [ 16 , 17 , 44 - 47 ], which encompasses non–face-to-face care models, mHealth, and occasionally AI-based solutions [ 18 ] within its scope. It is pertinent to mention that the differentiation between the methodological frameworks designed for the evaluation of eHealth and those designed for dHTA was based on the specific terminology and descriptions used by the authors of those frameworks.

The structures and characteristics of the analyzed methodological frameworks were considered heterogeneous in terms of evaluation specificity (whether they focused on a global evaluation that encompassed more than 1 domain or dimension or on a specific assessment that addressed only 1 domain or dimension), assessment approach (whether they adopted a phased evaluation, a domain evaluation, or a hybrid of both), and number of domains included. Regarding evaluation specificity, 17 (89.5%) methodological frameworks were classified as global as they covered various aspects or domains within their scope [ 16 , 17 , 33 - 36 , 38 - 47 , 55 , 56 ], while 2 (10.5%) were classified as specific as they concentrated exclusively on 1 element or domain of assessment [ 37 , 46 ]. Regarding the assessment approach, 14 (73.7%) methodological frameworks proposed a domain-based evaluation [ 16 , 17 , 33 , 35 , 36 , 38 - 40 , 43 , 44 , 46 , 55 , 56 ], while 4 (21.1%) proposed a hybrid one (phased and domain based) [ 41 , 42 , 45 , 47 ]; the remaining methodological framework did not fit into any of the previous categories, as it was not structured by domains or phases but by types of risk [ 37 ]. Finally, the number of evaluation domains considered ranged from 1 to 14, with an average of 7. Table 1 outlines the primary features of the included methodological frameworks and provides a thorough breakdown of the domains and dimensions they address.

In contrast, from 3 (75%) [ 49 - 51 ] of the 4 SRs [ 48 - 51 ] and the report from the working group on guidelines for the evaluation of mHealth solutions from the European mHealth Hub [ 52 ], we identified other methodological frameworks and tools focusing on the assessment of dHTs. Specifically, we identified 16 methodological frameworks or tools focusing on the evaluation of non–face-to-face care models [ 57 - 72 ], along with 37 for the evaluation of mHealth [ 10 , 52 , 73 - 95 ], 11 for the evaluation of eHealth [ 96 - 107 ], and 17 for the evaluation of dHTs in general [ 108 - 124 ]. Additionally, 5 (26.3%) [ 33 , 34 , 36 , 37 , 42 ] of the 19 methodological frameworks included in this ScR were also identified and analyzed in 1 or more of the 4 literature synthesis documents [ 49 - 52 ]. It is important to note that the difference between the frameworks we retrieved through our systematic search and those identified in the 4 SRs is the result of the narrower perspective we adopted, focusing exclusively on frameworks directly relevant to the HTA field, in line with the aims of our study. In Multimedia Appendix 4 , we provide a more detailed explanation of the methodological frameworks included in the studies mentioned earlier [ 19 , 49 - 52 , 57 - 73 , 75 - 135 ].

a ScR: scoping review.

b mHealth: mobile health.

c N/A: not applicable.

d AI: artificial intelligence.

e dHT: digital health technology.

Research Question 2: Domains and Dimensions Being Considered in dHTA

The 26 (32.1%) studies included encompassed a broad range of items to consider in dHTA and often used diverse expressions for analogous concepts. We reduced this heterogeneity through our thematic analysis according to the recommendations and phases described by Thomas and Harden [ 31 ].

In this sense, in the first phase of thematic analysis, we identified and coded 176 units of meaning (coded as provisional codes) that represented different items (domains or dimensions) of the assessment. These units were then grouped into 86 descriptive themes (second phase), which were further refined into 61 analytical themes that captured the key concepts and relationships between them (third phase). Lastly, the 61 analytical themes were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). We used the term “domain” to refer to a distinct area or topic of evaluation that is integral to the assessment of the technology in question. A domain may encompass multiple related concepts or dimensions that are relevant to the evaluation. Each dimension, in turn, represents a specific aspect of evaluation that belongs to the domain and contributes to an understanding of its overall significance. Finally, a subdimension refers to a partial element of a dimension that facilitates its analysis. By using these terms, we aimed to provide a clear, rigorous, and comprehensive framework for conducting HTA.

Table 2 displays the 61 analytical themes in descending order of coding frequency, aligned with the hierarchy derived from the data analysis. Additionally, the table specifies the intervention modalities or dHTs that correspond to each code and lists the studies from which each code originated. The network of relationships among the codes can be found in Multimedia Appendix 5 .

a dHT: digital health technology.

c AI: artificial intelligence.

d N/A: not applicable.

Research Question 3: Variability of Domains and Dimensions Among Technologies

Our thematic analysis revealed a significant degree of variability and heterogeneity in the number and type of domains and dimensions considered by the methodological frameworks.

In terms of numbers, the variability was quite pronounced when we compared frameworks addressing different types of dHTs. For instance, the thematic analysis of frameworks for assessing telemedicine only identified 9 (75%) domains and 6 (15.8%) dimensions; instead, in frameworks for assessing mHealth, we identified 10 (83.3%) domains, 20 (52.6%) dimensions, and 6 (54.5%) subdimensions, and in frameworks for assessing AI, we identified 8 (66.7%) different domains, 7 (18.4%) different dimensions, and 6 (54.5%) subdimensions.

In terms of the types of domains considered, certain dimensions and domains were identified as more distinctive for one kind of dHT than for another. For instance, clinical efficacy and effectiveness, technical safety, economic evaluation, and user experience were relevant for the evaluation of models of nonpresential health care and mHealth but not for AI. In contrast, there were specific dimensions and domains of mHealth that were not considered in the evaluation of non–face-to-face health care or AI, such as postmarketing monitoring, scientific basis, technical evaluation and validation, user control and self-determination, accessibility, content and adequacy of information, and data interoperability and integration. Finally, specific methodological frameworks for the evaluation of AI included dimensions such as technical aspects, adoption, use, integration, generalizability, reproducibility, and interpretability, which were not considered in the evaluation of telemedicine or mHealth. In conclusion, greater clarity and structuring in the presentation of these ideas are required to facilitate their understanding and assimilation.

Proposal for Domains, Dimensions, and Subdimensions for dHTA

These findings led to the development of a proposed methodological framework for dHTA, which comprises domains, dimensions, and subdimensions. These evaluation items were established objectively based on thematically analyzed evidence, without incorporating the researcher’s perspective. Consequently, the proposal for domains, dimensions, and subdimensions emerged from the literature and represents the entirety of identified evaluation domains, dimensions, and subdimensions (n=61). Figure 2 presents a visual representation of the proposed framework comprising 12 domains, 38 dimensions, and their corresponding 11 subdimensions. Notably, the figure highlights certain domains, dimensions, and subdimensions that are particularly relevant to the evaluation of non–face-to-face care models, mHealth, and AI according to the evidence.

Principal Findings

In recent years, the interest in digital health has increased significantly, giving rise to a myriad of available technologies. This has brought about a profound transformation in health care systems, fundamentally changing the provision and consumption of health care services [ 9 ]. However, despite these advancements, the shift toward digital health has been accompanied by challenges. One such challenge is the emergence of a plethora of short-lived implementations and an overwhelming diversity of digital tools, which has created a need for careful evaluation and analysis of the benefits and drawbacks of these technologies [ 4 ].

In this context, our ScR aimed to identify the methodological frameworks used worldwide for the assessment of dHTs; determine what domains are considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

Throughout the ScR, we identified a total of 95 methodological frameworks and tools, of which 19 [ 16 , 17 , 33 - 47 ] were directly identified through a systematic search and 75 were indirectly identified through 4 SRs [ 49 - 52 ]. The difference in the number of methodological frameworks identified through the ScR and the 4 evidence synthesis documents [ 49 - 52 ] is attributed to the inclusion of keywords related to the concept of HTA in the search syntax, the exclusion of methodological frameworks published prior to 2011 during the screening process, and the differences in perspectives used for the development of this paper compared to the 4 evidence synthesis documents mentioned earlier. In this sense, these 4 documents [ 49 - 52 ] have analyzed methodological frameworks and tools aimed at evaluating digital health that have not been developed from an HTA perspective despite the authors analyzing them as such. For example, von Huben et al. [ 51 ] included in their analysis the Consolidated Standards of Reporting Trials (CONSORT)-EHEALTH tool [ 97 ], which aims to describe the information that should be reported in papers and reports that focus on evaluating web- and mHealth-based interventions; Koladas et al [ 49 ] included the mobile health evidence reporting and assessment (mERA) checklist [ 73 ], which aims to determine the information that should be reported in trials evaluating mHealth solutions; and the European mHealth Hub document [ 52 ] includes the Isys Score, which is for cataloguing apps for smartphones.

However, as detailed in the Results section, some of the methodological frameworks identified through the ScR were characterized by the authors themselves as being specific for evaluating certain types of dHTs (eg, non–face-to-face care models, mHealth), presenting certain differences according to each typology. It is important to note that the differentiation among various types of dHTs, as described throughout this paper and commonly used in the field of digital health, cannot always be made in a precise and exclusive manner [ 136 ]. This is because a technology often can be classified in more than 1 category. For instance, an mHealth solution may use AI algorithms, while simultaneously being integrated into a non–face-to-face care model [ 137 ]. In this context, future research should consider using alternative taxonomies or classification methods that are based on the intended purpose of the technology, such as those proposed by NICE in the updated version of the Evidence Standards Framework [ 18 ] or the new digital health interventions system classification put forward by WHO [ 138 ].

After conducting a thematic analysis of the 26 included studies, we observed that various methodological frameworks include a set of evaluation items, referred to as domains, dimensions, or criteria. These items primarily focus on the safety; effectiveness; technical aspects; economic impact; and ethical, legal, and social consequences of dHTs. However, there is significant heterogeneity among these frameworks in terms of the way they refer to the evaluation items, the quantity and depth of their description, the degree of granularity, and the proposed evaluation methods, especially when comparing frameworks that focus on different types of dHTs. Despite this heterogeneity, most methodological frameworks consider evaluation items related to the 9 domains described by the HTA Core Model of EUnetHTA, while some frameworks propose additional evaluation elements, such as usability [ 16 , 44 , 45 , 47 , 49 , 56 ], privacy [ 39 - 41 , 44 , 52 , 55 ], and technical stability [ 16 , 38 , 47 , 49 , 52 ] among others. These findings are consistent with earlier research [ 50 , 51 ].

In addition, through the thematic analysis, the heterogeneity identified among the different methodological frameworks included in this ScR was reduced to a total of 61 analytical themes related to various evaluation elements that were arranged in a 3-level vertical hierarchy based on the evidence: level 1 (12 domains), level 2 (38 dimensions), and level 3 (11 subdimensions). At this point, it is pertinent to note that although from the researchers’ perspective, some dimensions could have been classified under different domains (eg, responsibility under ethical aspects) or seen as essential for other kinds of dHTs, an effort was made to maintain the highest degree of objectivity possible. It is for this reason that privacy issues were not described as essential for non–face-to-face care models and why the dimension of accessibility was categorized within the domains of human and sociocultural aspects and technical aspects. This categorization was made because some of the methodological frameworks analyzed associated it with sociocultural elements (eg, evaluating whether users with functional diversity can access the technology and have sufficient ability to use it as expected), while others linked it to technical elements (eg, adequacy of the elements, options, or accessibility functionalities that the system incorporates according to the target audience) [ 16 , 52 ].

The ScR and thematic analysis conducted in this study led to a proposal for a methodological framework for dHTA. This framework was further developed using additional methodologies, such as consensus workshops by the Agency for Health Quality and Assessment of Catalonia (AQuAS), in collaboration with all agencies of RedETS, commissioned by the Ministry of Health of Spain. The final framework is a specific methodological tool for the assessment of dHTs, aimed at describing the domains and dimensions to be considered in dHTA and defining the evidence standards that such technologies must meet based on their associated risk level. The proposed methodological framework enables the assessment of a wide range of dHTs, mainly those classified as medical devices according to the Regulation (EU) 2017/745 for medical devices [ 139 ] and Regulation (EU) 2017/746 for in vitro diagnostic medical devices, although it can be adapted to assess dHTs not classified as medical devices [ 140 ]. Unlike existing frameworks, it establishes a clear link between the identified domains and dimensions and the evidence standards required for dHTs to meet. This approach will enhance the transparency and consistency of dHTAs and support evidence-based decision-making. The final document was published from November 2023 onward and is available on the RedETS website as well as on the main web page of AQuAS in the Spanish language [ 141 ]. From the first week of February, the respective websites have hosted an English version of this document [ 141 ], which also is accessible in the INAHTA database. In addition, the Spanish and English versions of the document will be periodically reviewed and, if necessary, adapted to align with emerging technologies and changes in legislation.

Limitations

Although this ScR was conducted in accordance with the PRISMA-ScR guidelines ( Multimedia Appendix 1 ) and following the recommendations of Peters et al [ 22 ] and Pollock et al [ 23 ], there were some limitations. First, the search incorporated a block of keywords related to the concept of HTA (see Multimedia Appendix 1 ) due to the perspective of our ScR, which may have limited the retrieval of some studies to meet the study objective. However, this limitation was compensated for by the analysis of the 3 SRs and the report of the working group on guidelines for the evaluation of mHealth solutions of the European mHealth Hub. Second, much of the literature related to HTA is gray literature and only published on the websites of the authoring agencies. Despite efforts to address this limitation through expert input and a comprehensive search of the websites of the world’s leading agencies, it is possible that certain studies were not identified. Third, the quality and limitations of the analysis conducted by the authors of methodological frameworks and tools included in SRs may have had an impact on the indirect thematic analysis. Therefore, it is possible that some data could have been omitted or not considered during this process. Fourth, the focus on dHTs encompassed within the 3 previously mentioned categories (mHealth, non–face-to-face care models, and medical devices that integrate AI) may have influenced the outcomes of the thematic analysis conducted. Fifth, only methodological frameworks written in Catalan, Spanish, and English were included.

Comparison With Prior Work

To the best of our knowledge, this is the first ScR to examine the methodological frameworks for dHTA, followed by a thematic analysis with the aim of proposing a new comprehensive framework that incorporates the existing literature in an objective manner and enables the assessment of various technologies included under the concept of digital health. In this sense, existing SRs and other evidence synthesis documents have only analyzed the literature and reported the results in a descriptive manner [ 36 , 48 , 49 , 51 , 56 , 125 , 126 ]. Furthermore, this ScR also considered, in addition to scientific literature, gray literature identified by searching the websites of the agencies, thus covering some limitations of previous reviews [ 50 ]. Moreover, this review was carried out from the perspective of HTA, addressing a clear need expressed by HTA agencies [ 16 ].

Future research should aim to identify what domains and dimensions are relevant at the different stages of the technology life cycle, to establish or develop a standardized set of outcomes for assessing or reporting each domain, and to evaluate the effectiveness and usefulness of the existing methodological frameworks for the different intended users [ 50 , 142 ]. Moreover, future research should aim to determine the specific evaluation criteria that ought to be considered based on the level of risk associated with different types of technologies [ 51 ].

Our ScR revealed a total of 102 methodological frameworks and tools designed for evaluating dHTs, with 19 being directly identified through a systematic search and 83 through 4 evidence synthesis documents. Only 19 of all the identified frameworks were developed from the perspective of HTA. These frameworks vary in assessment items, structure, and specificity, and their proven usefulness in practice is scarce.

The thematic analysis of the 26 studies that met the inclusion criteria led to the identification and definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered when evaluating dHTs. Building on our results, a methodological framework for dHTA was proposed.

Acknowledgments

We acknowledge Benigno Rosón Calvo (Servicio Gallego de Salud [SERGAS]), Carme Carrion (Universitat Oberta de Catalunya [UOC]), Carlos A Molina Carrón (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Carme Pratdepadua (Fundació Tic Salut i Social [FTSS]), Celia Muñoz (Instituto Aragonés de Ciencias de la Salud [IACS]), David Pijoan (Biocat, BioRegió de Catalunya), Felip Miralles (Eurecat – Centre Tecnològic de Catalunya), Iñaki Guiterrez Ibarluzea (Osasun Teknologien Ebaluazioko Zerbitzua [Osteba]), Janet Puñal Riobóo (Unidad de Asesoramiento Científico-técnico [avalia-t], Agencia Gallega para la Gestión del Conocimiento en Salud [ACIS]), Jordi Piera-Jiménez (Àrea de Sistemes d’Informació del Servei Català de la Salut [CatSalut]), Juan Antonio Blasco (Evaluación de Tecnologías Sanitarias de Andalucía [AETSA]), Liliana Arroyo Moliner (Direcció General de Societat Digital, Departament d’Empresa i Treball de la Generalitat de Catalunya), Lilisbeth Perestelo-Perez (Servicio de Evaluación del Servicio Canario de la Salud [SESCS]), Lucía Prieto Remón (IACS), Marifé Lapeña (Dirección General de Salud Digital y Sistemas de Información para el SNS. Ministerio de Sanidad, Gobierno de España), Mario Cárdaba (Insituto de Salud Carlos III [ISCIII]), Montserrat Daban (Biocat, BioRegió de Catalunya), Montserrat Moharra Frances (Agència de Qualitat i Avaluació Sanitàries de Catalunya), and Oscar Solans (CatSalut) for reviewing the protocol of this scoping review (ScR) and the ScR.

This research was framed within the budget of the work plan of the Spanish Network of Health Technology Assessment Agencies, commissioned by the General Directorate of Common Portfolio of Services of the National Health System and Pharmacy.

Authors' Contributions

JS-F and CM-P were responsible for conceptualization, methodology, formal analysis, investigation, data curation, writing—original draft, and visualization. RP-P handled conceptualization, methodology, formal analysis, investigation, resources, and writing—original draft. RMV-H handled conceptualization, writing—review and editing, supervision, and project administration.

Conflicts of Interest

None declared.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews (PRISMA-ScR) checklist [ 21 ].

Search strategies for each database.

References excluded at the full-text screening stage.

Methodological frameworks included in systematic reviews.

Network of relationships among the codes.

High-resolution image of Figure 2.

• Avanzas P, Pascual I, Moris C. The great challenge of the public health system in Spain. J Thorac Dis. May 2017;9(Suppl 6):S430-S433. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Grubanov BS, Ghio D, Goujon A, Kalantaryan S, Belmonte M, Scipioni M. Health and long-term care workforce: demographic challenges and the potential contribution of migration and digital technology. Luxembourg. European Commission; 2021;2045-2322.
• World Health Organization (WHO). Health 2020: a European policy framework and strategy for the 21st century. Denmark. WHO; 2013.
• World Health Organization (WHO). WHO guideline: recommendations on digital interventions for health system strengthening. Geneva. WHO; Jun 6, 2019.
• Scholz N. Addressing health inequalities in the European Union. Luxembourg. European Parliamentary Research Service; 2020.
• Kickbusch I, Agrawal A, Jack A, Lee N, Horton R. Governing health futures 2030: growing up in a digital world—a joint The Lancet and Financial Times Commission. Lancet. Oct 2019;394(10206):1309. [ CrossRef ]
• Reeves JJ, Ayers JW, Longhurst CA. Telehealth in the COVID-19 era: a balancing act to avoid harm. J Med Internet Res. Feb 01, 2021;23(2):e24785. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Richardson S, Lawrence K, Schoenthaler AM, Mann D. A framework for digital health equity. NPJ Digit Med. Aug 18, 2022;5(1):119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• European Union. State of health in the EU: companion report 2021. Luxembourg. European Union; 2022.
• LATITUD: Anàlisi comparativa de models d'atenció no presencial en l'àmbit de la salut. TIC Salut Social. 2020. URL: https://ticsalutsocial.cat/wp-content/uploads/2021/07/latitud.pdf [accessed 2024-03-28]
• Global strategy on digital health 2020-2025. World Health Organization. Aug 18, 2021. URL: https://www.who.int/publications/i/item/9789240020924 [accessed 2024-03-28]
• Ming J, He Y, Yang Y, Hu M, Zhao X, Liu J, et al. Health technology assessment of medical devices: current landscape, challenges, and a way forward. Cost Eff Resour Alloc. Oct 05, 2022;20(1):54. [ FREE Full text ] [ CrossRef ] [ Medline ]
• O'Rourke B, Oortwijn W, Schuller T. The new definition of health technology assessment: a milestone in international collaboration. Int J Technol Assess Health Care. May 13, 2020;36(3):187-190. [ CrossRef ]
• EUnetHTA Joint Action 2 Work Package 8. HTA Core Model ® version 3.0 2016. EUnetHTA. 2016. URL: https://www.eunethta.eu/wp-content/uploads/2018/03/HTACoreModel3.0-1.pdf [accessed 2024-03-28]
• Puñal-Riobóo J, Baños, Varela LL, Castillo MM, Atienza MG, Ubago PR. Guía para la elaboración y adaptación de informes rápidos. Santiago de Compostela, Madrid. Agencia Gallega para la Gestión del Conocimientoto en Salud. Unidad de Asesoramiento Científico-técnico, avalia-t. Ministerio de Sanidad, Servicios Sociales e Igualdad; 2016.
• Haverinen J, Keränen N, Falkenbach P, Maijala A, Kolehmainen T, Reponen J. Digi-HTA: health technology assessment framework for digital healthcare services. FinJeHeW. Nov 02, 2019;11(4):326-341. [ CrossRef ]
• Hussain MS, Silvera-Tawil D, Farr-Wharton G. Technology assessment framework for precision health applications. Int J Technol Assess Health Care. May 26, 2021;37(1):e67. [ CrossRef ]
• National Institute for Health and Care Excellence (NICE). Evidence standards framework (ESF) for digital health technologies. Contract no.: 30 de septiembre. London. NICE; 2022.
• Evidence standards framework (ESF) for digital health technologies. National Institute for Health and Care Excellence. URL: https:/​/www.​nice.org.uk/​about/​what-we-do/​our-programmes/​evidence-standards-framework-for-digital-health-technologies [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo RM. Methodological frameworks and dimensions to be taken into consideration in digital health technology assessment: protocol for a scoping review. JMIR Res Protoc. Oct 11, 2022;11(10):e39905. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tricco AC, Lillie E, Zarin W, O'Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. Oct 02, 2018;169(7):467-473. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Peters M, Marnie C, Tricco A, Pollock D, Munn Z, Alexander L. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18(10):2119-2126. [ CrossRef ]
• Pollock D, Davies EL, Peters MDJ, Tricco AC, Alexander L, McInerney P, et al. Undertaking a scoping review: a practical guide for nursing and midwifery students, clinicians, researchers, and academics. J Adv Nurs. Apr 04, 2021;77(4):2102-2113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Ayiku L, Hudson T, Glover S, Walsh N, Adams R, Deane J, et al. The NICE MEDLINE and Embase (Ovid) health apps search filters: development of validated filters to retrieve evidence about health apps. Int J Technol Assess Health Care. Oct 27, 2020;37(1):e16. [ CrossRef ]
• McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. Jul 2016;75:40-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. Dec 06, 2017;6(1):245. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. 2015;13(3):179-187. [ CrossRef ]
• Systematic reviews - research guide. Defining your review question. Murdoch University. 2023. URL: https://libguides.murdoch.edu.au/systematic [accessed 2024-03-28]
• Scientific Software Development GmbH. Atlas.ti qualitative data analysis. 22.0 ed. Berlin. Scientific Software Development GmbH; 2021.
• Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions version 6.3. London. Cochrane; 2022.
• Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. Jul 10, 2008;8(1):45. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Page M, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. PLoS Med. Mar 2021;18(3):e1003583. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: MAST. Int J Technol Assess Health Care. Jan 23, 2012;28(1):44-51. [ CrossRef ]
• Nepal S, Li J, Jang-Jaccard J, Alem L. A framework for telehealth program evaluation. Telemed J E Health. Apr 2014;20(4):393-404. [ CrossRef ] [ Medline ]
• Measuring the impact of telehealth and telecare: SCTT toolkit. Glasgow. SCTT; 2013.
• Caulfield B, Reginatto B, Slevin P. Not all sensors are created equal: a framework for evaluating human performance measurement technologies. NPJ Digit Med. Feb 14, 2019;2(1):7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lewis TL, Wyatt JC. mHealth and mobile medical apps: a framework to assess risk and promote safer use. J Med Internet Res. Sep 15, 2014;16(9):e210. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Moshi MR, Tooher R, Merlin T. Development of a health technology assessment module for evaluating mobile medical applications. Int J Technol Assess Health Care. May 18, 2020;36(3):252-261. [ CrossRef ]
• Puigdomènech E, Poses-Ferrer E, Espallargues CM, Blasco AJ, Varela LL, Paz VL. Evaluación de tecnología basada en mSalud para aplicaciones móviles. Madrid. Ministerio de Sanidad; 2020.
• Henson P, David G, Albright K, Torous J. Deriving a practical framework for the evaluation of health apps. Lancet Digit Health. Jun 2019;1(2):e52-e54. [ CrossRef ]
• Reddy S, Rogers W, Makinen V, Coiera E, Brown P, Wenzel M, et al. Evaluation framework to guide implementation of AI systems into healthcare settings. BMJ Health Care Inform. Oct 12, 2021;28(1):e100444. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Khoja S, Durrani H, Scott RE, Sajwani A, Piryani U. Conceptual framework for development of comprehensive e-health evaluation tool. Telemed J E Health. Jan 2013;19(1):48-53. [ CrossRef ] [ Medline ]
• Sockolow P, Bowles K, Rogers M. Health information technology evaluation framework (HITREF) comprehensiveness as assessed in electronic point-of-care documentation systems evaluations. Amsterdam. IOS Press; 2015.
• Kowatsch T, Otto L, Harperink S, Cotti A, Schlieter H. A design and evaluation framework for digital health interventions. it - Inf Technol. Nov 2019;61(5):253-263. [ CrossRef ]
• Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: a path to validation. NPJ Digit Med. May 13, 2019;2(1):38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Milosevic Z. Ethics in digital health: a deontic accountability framework. 2019. Presented at: IEEE 23rd International Enterprise Distributed Object Computing Conference (EDOC); October 28-31, 2019; Paris, France. [ CrossRef ]
• World Health Organization (WHO). Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva. WHO; 2016.
• Enam A, Torres-Bonilla J, Eriksson H. Evidence-based evaluation of eHealth interventions: systematic literature review. J Med Internet Res. Nov 23, 2018;20(11):e10971. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kolasa K, Kozinski G. How to value digital health interventions? A systematic literature review. Int J Environ Res Public Health. Mar 23, 2020;17(6):2119. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vis C, Bührmann L, Riper H, Ossebaard HC. Health technology assessment frameworks for eHealth: a systematic review. Int J Technol Assess Health Care. Apr 16, 2020;36(3):204-216. [ CrossRef ]
• von Huben A, Howell M, Howard K, Carrello J, Norris S. Health technology assessment for digital technologies that manage chronic disease: a systematic review. Int J Technol Assess Health Care. May 26, 2021;37(1):e66. [ CrossRef ]
• Report of the Working Group on mhealth assessment guidelines. European Commission. 2021. URL: https://digital-strategy.ec.europa.eu/en/library/report-working-group-mhealth-assessment-guidelines [accessed 2024-03-28]
• Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, et al. Mobile health technology evaluation: the mhealth evidence workshop. Am J Prev Med. Aug 2013;45(2):228-236. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alami H, Lehoux P, Auclair Y, de Guise M, Gagnon M, Shaw J, et al. Artificial intelligence and health technology assessment: anticipating a new level of complexity. J Med Internet Res. Jul 07, 2020;22(7):e17707. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Vokinger KN, Nittas V, Witt CM, Fabrikant SI, von Wyl V. Digital health and the COVID-19 epidemic: an assessment framework for apps from an epidemiological and legal perspective. Swiss Med Wkly. May 04, 2020;150(1920):w20282. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Baumel A, Muench F. Heuristic evaluation of ehealth interventions: establishing standards that relate to the therapeutic process perspective. JMIR Ment Health. Jan 13, 2016;3(1):e5. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alfonzo A, Huerta M, Wong S, Passariello G, Diaz M, La CA, et al. Design of a methodology for assessing an electrocardiographic telemonitoring system. Annu Int Conf IEEE Eng Med Biol Soc. 2007;2007:3729-3732. [ CrossRef ]
• Bashshur R, Shannon G, Sapci H. Telemedicine evaluation. Telemed J E Health. Jun 2005;11(3):296-316. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Beintner I, Vollert B, Zarski A, Bolinski F, Musiat P, Görlich D, et al. Adherence reporting in randomized controlled trials examining manualized multisession online interventions: systematic review of practices and proposal for reporting standards. J Med Internet Res. Aug 15, 2019;21(8):e14181. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brear M. Evaluating telemedicine: lessons and challenges. Health Inf Manag. Jul 21, 2006;35(2):23-31. [ CrossRef ] [ Medline ]
• DeChant HK, Tohme WG, Mun SK, Hayes WS, Schulman KA. Health systems evaluation of telemedicine: a staged approach. Telemed J. Jan 1996;2(4):303-312. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part II: tools for a quality control system. Telemed J E Health. Apr 2007;13(2):130-140. [ CrossRef ] [ Medline ]
• Giansanti D, Morelli S, Macellari V. Telemedicine technology assessment part I: setup and validation of a quality control system. Telemed J E Health. Apr 2007;13(2):118-129. [ CrossRef ] [ Medline ]
• Grigsby J, Brega AG, Devore PA. The evaluation of telemedicine and health services research. Telemed J E Health. Jun 2005;11(3):317-328. [ CrossRef ] [ Medline ]
• Hailey D, Jacobs P, Simpson J, Doze S. An assessment framework for telemedicine applications. J Telemed Telecare. Jun 23, 1999;5(3):162-170. [ CrossRef ] [ Medline ]
• Ohinmaa A, Hailey D, Roine R. Elements for assessment of telemedicine applications. Int J Technol Assess Health Care. Jun 30, 2001;17(2):190-202. [ CrossRef ] [ Medline ]
• Rajan B, Tezcan T, Seidmann A. Service systems with heterogeneous customers: investigating the effect of telemedicine on chronic care. Manag Sci. Mar 2019;65(3):1236-1267. [ CrossRef ]
• Sisk JE, Sanders JH. A proposed framework for economic evaluation of telemedicine. Telemed J. Jan 1998;4(1):31-37. [ CrossRef ] [ Medline ]
• Zissman K, Lejbkowicz I, Miller A. Telemedicine for multiple sclerosis patients: assessment using Health Value Compass. Mult Scler. Apr 30, 2012;18(4):472-480. [ CrossRef ] [ Medline ]
• Grustam AS, Vrijhoef HJM, Koymans R, Hukal P, Severens JL. Assessment of a business-to-consumer (B2C) model for telemonitoring patients with chronic heart failure (CHF). BMC Med Inform Decis Mak. Oct 11, 2017;17(1):145. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hebert M. Telehealth success: evaluation framework development. Stud Health Technol Inform. 2001;84(Pt 2):1145-1149. [ Medline ]
• Rojahn K, Laplante S, Sloand J, Main C, Ibrahim A, Wild J, et al. Remote monitoring of chronic diseases: a landscape assessment of policies in four European countries. PLoS One. May 19, 2016;11(5):e0155738. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Agarwal S, LeFevre AE, Lee J, L'Engle K, Mehl G, Sinha C, et al. WHO mHealth Technical Evidence Review Group. Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ. Mar 17, 2016;352:i1174. [ CrossRef ] [ Medline ]
• Safety and quality strategy in mobile health apps. Complete list of recommendations on design, use and assessment of health apps. Agencia de Calidad Sanitaria de Andalucía. 2012. URL: http://www.calidadappsalud.com/en/listado-completo-recomendaciones-app-salud/ [accessed 2023-07-17]
• The app evaluation model. American Psychiatric Association Initiative. 2022. URL: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/the-app-evaluation-model [accessed 2024-03-28]
• Gdd AppStore. Association of Regional Public Health Services (GGD) and Regional Medical Emergency Preparedness and Planning (GHOR). 2016. URL: https://tinyurl.com/58th4p4w [accessed 2023-07-18]
• AppQ: quality criteria core set for more quality transparency in digital health applications. Bertelsmann Stiftung. Oct 29, 2019. URL: https://www.bertelsmann-stiftung.de/de/publikationen/publikation/did/appq [accessed 2024-03-28]
• Bradway M, Carrion C, Vallespin B, Saadatfard O, Puigdomènech E, Espallargues M, et al. mHealth assessment: conceptualization of a global framework. JMIR Mhealth Uhealth. May 02, 2017;5(5):e60. [ FREE Full text ] [ CrossRef ] [ Medline ]
• PAS 277:2015 - health and wellness apps. Quality criteria across the life cycle. British Standards Institution. URL: https:/​/knowledge.​bsigroup.com/​products/​health-and-wellness-apps-quality-criteria-across-the-life-cycle-code-of-practice/​standard [accessed 2023-07-18]
• MindApps. Centre for Telepsychiatry in the Region of Southern Denmark. 2017. URL: https://mindapps.dk/ [accessed 2023-07-18]
• Dick S, O'Connor Y, Thompson MJ, O'Donoghue J, Hardy V, Wu TJ, et al. Considerations for improved mobile health evaluation: retrospective qualitative investigation. JMIR Mhealth Uhealth. Jan 22, 2020;8(1):e12424. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Federal Institute for Drugs and Medical Devices (BfArM). The fast-track process for digital health applications (diga) according to section 139e sgb v. A guide for manufacturers, service providers and users. Bonn, Germany. Federal Institute for Drugs and Medical Devices; 2020.
• Gorski I, Bram JT, Sutermaster S, Eckman M, Mehta K. Value propositions of mHealth projects. J Med Eng Technol. Aug 12, 2016;40(7-8):400-421. [ CrossRef ] [ Medline ]
• Hogaboam L, Daim T. Technology adoption potential of medical devices: the case of wearable sensor products for pervasive care in neurosurgery and orthopedics. Health Policy Technol. Dec 2018;7(4):409-419. [ CrossRef ]
• Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. Apr 05, 2019;2(4):e192542. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Maar MA, Yeates K, Perkins N, Boesch L, Hua-Stewart D, Liu P, et al. A framework for the study of complex mhealth interventions in diverse cultural settings. JMIR Mhealth Uhealth. Apr 20, 2017;5(4):e47. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McMillan B, Hickey E, Patel MG, Mitchell C. Quality assessment of a sample of mobile app-based health behavior change interventions using a tool based on the National Institute of Health and Care Excellence behavior change guidance. Patient Educ Couns. Mar 2016;99(3):429-435. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Health and welness apps: new international guidelines to help to sort the best form the rest. NEN. URL: https://www.nen.nl/en/health-and-welness-apps [accessed 2024-03-28]
• Continua design guidelines. Personal Connected Health Alliance. 2019. URL: https://www.pchalliance.org/continua-design-guidelines [accessed 2023-07-18]
• Philpott D, Guergachi A, Keshavjee K. Design and validation of a platform to evaluate mhealth apps. Stud Health Technol Inform. 2017;235:3-7. [ Medline ]
• Ruck A, Wagner BS, Lowe C. Second draft of guidelines. EU guidelines on assessment of the reliability of mobile health applications. European Commission, Directorate-General of Communications Networks, Content & Technology. Luxembourg; 2016.
• Sax M, Helberger N, Bol N. Health as a means towards profitable ends: mhealth apps, user autonomy, and unfair commercial practices. J Consum Policy. May 22, 2018;41(2):103-134. [ CrossRef ]
• Wyatt JC. How can clinicians, specialty societies and others evaluate and improve the quality of apps for patient use? BMC Med. Dec 03, 2018;16(1):225. [ FREE Full text ] [ CrossRef ] [ Medline ]
• IRBs could address ethical issues related to tracking devices: mobile devices raise new concerns. IRB Advisor. Nov 1, 2017. URL: https:/​/www.​reliasmedia.com/​articles/​141589-irbs-could-address-ethical-issues-related-to-tracking-devices [accessed 2024-03-28]
• App check. Zentrum für Telematik und Telemedizin. URL: https://ztg-nrw.de/ [accessed 2024-03-28]
• Bergmo TS. How to measure costs and benefits of ehealth interventions: an overview of methods and frameworks. J Med Internet Res. Nov 09, 2015;17(11):e254. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Eysenbach G, CONSORT-EHEALTH Group. CONSORT-EHEALTH: improving and standardizing evaluation reports of web-based and mobile health interventions. J Med Internet Res. Dec 31, 2011;13(4):e126. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shaw NT. 'CHEATS': a generic information communication technology (ICT) evaluation framework. Comput Biol Med. May 2002;32(3):209-220. [ CrossRef ] [ Medline ]
• Brown M, Shaw N. Evaluation practices of a major Canadian telehealth provider: lessons and future directions for the field. Telemed J E Health. Oct 2008;14(8):769-774. [ CrossRef ] [ Medline ]
• Casper GR, Kenron DA. A framework for technology assessment: approaches for the selection of a home technology device. Clin Nurse Spec. 2005;19(4):170-174. [ CrossRef ] [ Medline ]
• Sitting D, Kahol K, Singh H. Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India. Stud Health Technol Inform. 2013;192:515-519. [ CrossRef ]
• Haute Autorité de Santé. Good practice guidelines on health apps and smart devices (mobile health or mhealth). Paris. Haute Autorité de Santé; Nov 7, 2016.
• Health Information and Quality Authority. International review of consent models for the collection, use and sharing of health information. Dublin. Health Information and Quality Authority; 2020.
• Jurkeviciute M. Planning of a holistic summative ehealth evaluation: the interplay between standards and reality PhD Thesis. Gotherburg. Chalmers University of Technology; 2018.
• Vimarlund V, Davoody N, Koch S. Steps to consider for effective decision making when selecting and prioritizing eHealth services. Stud Health Technol Inform. 2013;192:239-243. [ Medline ]
• Currie WL. TEMPEST: an integrative model for health technology assessment. Health Policy Technol. Mar 2012;1(1):35-49. [ CrossRef ]
• Eivazzadeh S, Anderberg P, Larsson TC, Fricker SA, Berglund J. Evaluating health information systems using ontologies. JMIR Med Inform. Jun 16, 2016;4(2):e20. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Our data-driven future in healthcare: people and partnerships at the heart of health related technologies. Academy of Medical Sciences. Nov 2018. URL: https://acmedsci.ac.uk/file-download/74634438 [accessed 2024-03-28]
• Australian Commission on Safety and Quality in Healthcare. National safety and quality digital mental health standards - consultation draft. Australia. Australian Commission on Safety and Quality in Healthcare; 2020.
• A guide to good practice for digital and data-driven health technologies. London. Department of Health & Social Care; 2021.
• Drury P, Roth S, Jones T, Stahl M, Medeiros D. Guidance for investing in digital health. In: Sustainable Development Working Papers. Mandaluyong, the Philippines. Asian Development Bank; May 2018.
• European Commission. Synospis report. Consultation: transformation health and care in the digital single market. Luxembourg. European Commission; 2018.
• Federal Ministry of Health. Regulation on the procedure and requirements for testing the eligibility for reimbursement of digital health applications in the statutory public health insurance (Digital Health Applications Ordinance - DiGAV). Alemania. Federal Ministry of Health; 2020.
• Haute Autorité de Santé. Guide to the specific features of clinical evaluation of a connected medical device (CMD) in view of its application for reimbursement. Paris. Haute Autorité de Santé; 2019.
• How we assess health apps and digital tools. NHS Digital. 2019. URL: https:/​/digital.​nhs.uk/​services/​nhs-apps-library/​guidance-forhealth-app-developers-commissioners-and-assessors/​how-we-assess-healthapps-and-digital-tools [accessed 2023-07-17]
• Lennon MR, Bouamrane M, Devlin AM, O'Connor S, O'Donnell C, Chetty U, et al. Readiness for delivering digital health at scale: lessons from a longitudinal qualitative evaluation of a national digital health innovation program in the United Kingdom. J Med Internet Res. Feb 16, 2017;19(2):e42. [ FREE Full text ] [ CrossRef ] [ Medline ]
• McNamee P, Murray E, Kelly MP, Bojke L, Chilcott J, Fischer A, et al. Designing and undertaking a health economics study of digital health interventions. Am J Prev Med. Nov 2016;51(5):852-860. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Haute Autorité de Santé. Medical Device and Health Technology Evaluation Committee (CNEDiMTS*). Paris. Haute Autorité de Santé; 2019.
• Draft guidelines for preparing assessment reports for the Medical Services Advisory Committee: draft version 4.0. Australian Government Department of Health and Aged Care. Aug 2020. URL: https:/​/consultations.​health.gov.au/​technology-assessment-access-division/​msac-guidelines-review-consultation/​user_uploads/​draft-msac-guidelines---clean-version---28-august-2020-3.​pdf [accessed 2024-03-28]
• Haute Autorité de Santé. Methodological choices for the clinical development of medical devices. Paris. Haute Autorité de Santé; 2013.
• Michie S, Yardley L, West R, Patrick K, Greaves F. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res. Jun 29, 2017;19(6):e232. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Mohr DC, Schueller SM, Riley WT, Brown CH, Cuijpers P, Duan N, et al. Trials of intervention principles: evaluation methods for evolving behavioral intervention technologies. J Med Internet Res. Jul 08, 2015;17(7):e166. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Murray E, Hekler EB, Andersson G, Collins LM, Doherty A, Hollis C, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med. Nov 2016;51(5):843-851. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Steventon A, Grieve R, Bardsley M. An approach to assess generalizability in comparative effectiveness research: a case study of the whole systems demonstrator cluster randomized trial comparing telehealth with usual care for patients with chronic health conditions. Med Decis Making. May 18, 2015;35(8):1023-1036. [ CrossRef ]
• D2.1 Knowledge Tool 1. Health apps assessment frameworks. European mHealth Hub. 2020. URL: https://mhealth-hub.org/download/d2-1-knowledge-tool-1-health-apps-assessment-frameworks [accessed 2024-03-28]
• Moshi MR, Tooher R, Merlin T. Suitability of current evaluation frameworks for use in the health technology assessment of mobile medical applications: a systematic review. Int J Technol Assess Health Care. Sep 11, 2018;34(5):464-475. [ CrossRef ]
• Stensgaard T, Sørensen T. Telemedicine in Greenland — the creation of an evaluation plan. J Telemed Telecare. Jun 22, 2016;7(1_suppl):37-38. [ CrossRef ]
• HL7 Consumer Mobile Health Application Functional Framework (cMHAFF). HL7 International. 2018. URL: http://www.hl7.org/implement/standards/product_brief.cfm?product_id=476 [accessed 2024-03-28]
• mHealth. Kantonen K-uKvBu. 2017. URL: https://www.e-health-suisse.ch/gemeinschaften-umsetzung/ehealth-aktivitaeten/mhealth.html [accessed 2023-07-18]
• mHealthBelgium. mHealthBELGIUM. URL: https://mhealthbelgium.be/ [accessed 2024-03-28]
• Mookherji S, Mehl G, Kaonga N, Mechael P. Unmet need: improving mhealth evaluation rigor to build the evidence base. J Health Commun. Jun 04, 2015;20(10):1224-1229. [ CrossRef ] [ Medline ]
• MySNS Selecção. Serviços Partilhados do Minisério da Saúde. URL: https://mysns.min-saude.pt/mysns-seleccao-processo-de-avaliacao/ [accessed 2023-07-18]
• Nielsen S, Rimpiläinen S. Report on international practice on digital apps. Glasgow. Digital Health & Care Institute; 2018.
• Accreditation of Digital Health solutions is a fundamental foundation for their safe adoption, equipping healthcare providers and practitioners with access to health apps assured to your standards Internet. Organisation for the Review of Care and Health Applications (ORCHA). URL: https://orchahealth.com/services/ [accessed 2023-07-18]
• mHealth. TIC Salut Social. 2022. URL: https://ticsalutsocial.cat/projecte/mhealth/ [accessed 2024-03-28]
• Wienert J, Jahnel T, Maaß L. What are digital public health interventions? First steps toward a definition and an intervention classification framework. J Med Internet Res. Jun 28, 2022;24(6):e31921. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deniz-Garcia A, Fabelo H, Rodriguez-Almeida AJ, Zamora-Zamorano G, Castro-Fernandez M, Alberiche Ruano MDP, et al. WARIFA Consortium. Quality, usability, and effectiveness of mhealth apps and the role of artificial intelligence: current scenario and challenges. J Med Internet Res. May 04, 2023;25:e44030. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Health Organization (WHO). Classification of digital interventions, services and applications in health: a shared language to describe the uses of digital technology for health, 2nd ed. Geneva. WHO; Oct 24, 2023.
• European Parliament, Council of the European Union. Regulation (EU) 2017/745 of the European Parliament and of the Council of 5 April 2017 on medical devices, amending Directive 2001/83/EC, Regulation (EC) No 178/2002 and Regulation (EC) No 1223/2009 and repealing Council Directives 90/385/EEC and 93/42/EEC. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32017R0745 [accessed 2024-03-28]
• European Parliament, Council of the European Union. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU. EUR-Lex. 2017. URL: https://eur-lex.europa.eu/eli/reg/2017/746/oj [accessed 2024-03-28]
• Segur-Ferrer J, Moltó-Puigmartí C, Pastells-Peiró R, Vivanco-Hidalgo R. Marco de evaluación de tecnologías sanitarias: adaptación para la evaluación de tecnologías de salud digital. Madrid, Barcelona. Ministerio de Sanidad, Agència de Qualitat i Avaluació Sanitàries de Catalunya; 2023.
• Benedetto V, Filipe L, Harris C, Spencer J, Hickson C, Clegg A. Analytical frameworks and outcome measures in economic evaluations of digital health interventions: a methodological systematic review. Med Decis Making. Oct 19, 2022;43(1):125-138. [ CrossRef ]

Abbreviations

Edited by T Leung; submitted 03.05.23; peer-reviewed by R Gorantla, KL Mauco, M Aymerich, J Haverinen, M Behzadifar; comments to author 10.11.23; revised version received 01.12.23; accepted 20.02.24; published 10.04.24.

©Joan Segur-Ferrer, Carolina Moltó-Puigmartí, Roland Pastells-Peiró, Rosa Maria Vivanco-Hidalgo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.04.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Open access
• Published: 01 December 2023

Compassion fatigue in healthcare providers: a scoping review

• Anna Garnett 1 ,
• Lucy Hui 2 ,
• Christina Oleynikov 1 &
• Sheila Boamah 3  

BMC Health Services Research volume  23 , Article number:  1336 ( 2023 ) Cite this article

2732 Accesses

2 Altmetric

Metrics details

The detrimental impacts of COVID-19 on healthcare providers’ psychological health and well-being continue to affect their professional roles and activities, leading to compassion fatigue. The purpose of this review was to identify and summarize published literature on compassion fatigue among healthcare providers and its impact on patient care. Six databases were searched: MEDLINE (Ovid), PsycINFO (Ovid), Embase (Ovid), CINAHL, Scopus, Web of Science, for studies on compassion fatigue in healthcare providers, published in English from the peak of the pandemic in 2020 to 2023. To expand the search, reference lists of included studies were hand searched to locate additional relevant studies. The studies primarily focused on nurses, physicians, and other allied health professionals. This scoping review was registered on Open Science Framework (OSF), using the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) extension to scoping review. From 11,715 search results, 24 met the inclusion criteria. Findings are presented using four themes: prevalence of compassion fatigue; antecedents of compassion fatigue; consequences of compassion fatigue; and interventions to address compassion fatigue. The potential antecedents of compassion fatigue are grouped under individual-, organization-, and systems-level factors. Our findings suggest that healthcare providers differ in risk for developing compassion fatigue in a country-dependent manner. Interventions such as increasing available personnel helped to minimize the occurrence of compassion fatigue. This scoping review offers important insight on the common causes and potential risks for compassion fatigue among healthcare providers and identifies potential strategies to support healthcare providers’ psychological health and well-being.

• What do we already know about this topic? The elevated and persistent mental stress associated with the COVID-19 pandemic predisposed healthcare providers (HCP) to various psychological conditions such as compassion fatigue. Declines in health providers’ mental health has been observed to negatively impact their professional performance and the quality of patient care.

• How does your research contribute to the field? This review provides an overview of the prevalence of compassion fatigue among HCPs across the globe during the COVID-19 pandemic. The main risk factors for compassion fatigue include younger age, female sex, being either a physician or a nurse, high workload, extensive work hours, and limited access to personal protective equipment (PPE). Negative behavioral intention towards patients has been identified to be a consequence of compassion fatigue. Interventions such as the provision of emotional support, increased monitoring for conditions such as stress and burnout, and increasing available personnel helped to minimize the occurrence of compassion fatigue.

• What are your research’s implications towards theory, practice, or policy? While the public health emergency associated with the COVID-19 pandemic has ended, the impact on human health resources persists. The findings of this review can inform policy decisions and implementation of evidence-based strategies to prevent, manage, and lessen the negative effects of compassion fatigue on HCPs and its subsequent impacts on patient care.

Peer Review reports

Introduction

The 2019-novel coronavirus disease (COVID-19) outbreak spread rapidly and by January 30 th , 2022 was formally proclaimed a global health emergency despite being first identified just over a month prior [ 1 ]. Although there have been five other global health emergencies associated with disease outbreaks since 2009, none has matched the scale and scope of the COVID-19 pandemic [ 2 ]. In the short-term the rapid increase in patients requiring acute care services presented unprecedented challenges for health systems. Care provision and infection control strategies were hampered by capacity limitations, staffing shortfalls and supply chain challenges [ 3 ]. As a result, healthcare providers (HCPs) encountered mounting levels of strain which have continued with little reprieve for the duration of and beyond the global COVID-19 pandemic. Limited access to personal protective equipment (PPEs) exacerbated transmission of the virus, compounding healthcare providers’ fears of contracting and spreading COVID-19 among their peers, patients and families [ 4 , 5 , 6 , 7 ]. HCPs also contracted COVID-19, became seriously ill and died with global estimates of HCP death between January 2020 and May 2021 being over 100,000. With time, the number of absences, extended sick leaves and staff turnovers increased [ 7 , 8 ]. The combination of short staffing, frequent changes to workflow and continuous care provision to patients who were gravely ill and had high mortality amplified the toll on health care providers [ 8 , 9 ]. While no longer a global health emergency, there continue to be COVID-19 cases and deaths. As of July 14, 2023 there were 767,972,961 COVID-19 cases and 6,950,655 deaths globally [ 10 ].

HCPs around the globe who treated severe COVID-19 cases, a process which necessitated in-depth compassionate engagement, became vulnerable to developing compassion fatigue as a result of their continued and in-depth involvement in the care of these severely ill patients and their families [ 11 ]. Compassion fatigue is defined as a composite of two measurements: burnout (sustained employment-related stress that compromises an individual’s desire to work) and secondary trauma (the development of traumatic symptoms resulting from the protracted exposure to the suffering of others) [ 12 , 13 ]. An individual experiencing compassion fatigue has a reduced ability for showing compassion to others, resulting from the prolonged exposure to witnessing the suffering of others without being able to relieve one’s anguish despite having the desire to do so [ 9 ]. Individuals experiencing compassion fatigue may express a range of behaviors such as increased work absences or declines in the ability to engage in work-related tasks such as decision-making. Burnout and secondary trauma are suggested to be mediated by compassion satisfaction—the pleasure that comes from helping behavior [ 11 , 12 ].

As the pandemic shifts from being a global health emergency to an endemic disease, there continues to be concern for HCP health and well-being [ 14 , 15 , 16 ]. The increased and chronic nature of the stress experienced during and beyond the COVID-19 pandemic has heightened HCPs risk for a range of negative psychological impacts such as depression, fearfulness, grief and post-traumatic stress disorder (PTSD) [ 17 ]. Prior infectious disease outbreaks (SARS-CoV-1, H1N1, MERS-CoV, Ebola) are also associated with an increased prevalence of declining mental health in HCPs [ 18 ]. A growing body of research on the COVID-19 pandemic highlights the range of psychological symptoms HCPs developed following their sustained exposure to COVID-19 including burnout, feelings of isolation, insomnia, grief, emotional exhaustion, depression, post-traumatic stress and depersonalization, some of which have persisted over time [ 14 , 17 , 19 , 20 , 21 , 22 ]. The consequences of HCPs’ declining psychological health and well-being has had impacts on the quality of patient care and indirectly on patient outcomes through inadequate staffing [ 18 ]. Compromises in HCPs’ ability to provide optimal clinical care can have serious consequences, including the worsening of patient conditions and the increased transmission of the infection from patients to others in the hospital [ 18 ]. In addition, compassion fatigue may be exacerbated by the COVID-19 pandemic, potentially leading to moral injury, decreased productivity, increased turnover, and reduced quality of care [ 23 ]. Moreover, a growing body of literature suggests that challenges across health systems will persist although COVID-19 is no longer a global health emergency [ 24 , 25 ]. As such, it is important to have a fulsome understanding of COVID-19’s toll on HCPs and tailor health system strategies accordingly.

As health care systems continue to experience a health human resources crisis, it is important to identify and understand the prevalence of compassion fatigue, identify contributing factors, and increase understanding of the consequences and actions that can be taken to address compassion fatigue among HCPs. While there has been in an increase in the body of published literature on the health and well-being of HCPs since the onset of the COVID-19 pandemic, there continues to be a knowledge gap mapping the incidence of compassion fatigue, its resultant impact on HCP well-being, and its potential influence on patient care provision [ 11 , 17 ]. A comprehensive review of the literature on compassion fatigue among HCPs can inform policy and practice initiatives to improve the current health human resources crisis experienced by many health systems. It may also aid in identifying prospective research foci.

The purpose of this scoping review was to synthesize and provide a synopsis of the literature on compassion fatigue among HCPs during the COVID-19 pandemic and to understand its broader impact. The review was guided by the following question: What is the current state of knowledge on compassion fatigue among HCPs over the course of COVID-19?

Project registration

This scoping review was registered under Open Science Framework. A project outline was submitted including the study hypotheses, design, and data collection procedures. The DOI for the registered project is as follows: https://doi.org/10.17605/OSF.IO/F4T7N . In addition, a scoping review protocol for this review has been published in a peer-reviewed journal ( https://doi.org/10.1136/bmjopen-2022-069843 ).

Study design

A systematic scoping review strategy was chosen to explore the existing body of literature pertaining to the research topic. The objective of a scoping review is to identify relevant literature on a given topic, without focusing on evaluating research quality or conducting a thorough analysis of selected studies, as systematic reviews typically do. Current gaps in research and directions for future research can be identified by means of summarizing emerging literature on compassion fatigue in HCPs.

The current scoping review used two methodological tools, namely the Arksey and O’Mally scoping review framework as well as the Joanna Briggs Institute Critical Appraisal Tools. The Arksey and O’Malley framework comprises five stages, which include: (1) formulating the research question; (2) identifying relevant studies; (3) selecting studies for inclusion; (4) extracting and organizing the data; and (5) collating, summarizing, and reporting the findings [ 26 ]. While scoping reviews typically do not require article appraisal, all articles were evaluated by one author (CO) using the methodology established by the Joanna Briggs Institute (JBI) to enhance the overall quality of the review [ 27 ]. No articles were excluded based on their quality, in accord with the Arksey and O’Malley framework [ 26 ].

Stage I: Identifying the research question(s)

The research objective and question were drafted by the authors (AG, LH, CO, SB) and can be found in the previous section under “Research aim”.

Stage II: Identifying relevant studies

As outlined by the JBI methodology, a three-step approach was used to identify relevant studies. These steps include: (1) conducting a preliminary search of at least two suitable databases; (2) identifying relevant keywords and index terms to perform a secondary search across all chosen databases; and (3) manually examining the reference lists of the included articles to discover additional relevant studies [ 28 ]   (p11) .

Preliminary literature search

To establish the criteria for inclusion and exclusion, an initial and restricted search was conducted on the subject of interest. The preliminary literature exploration encompassed three scholarly electronic databases: MEDLINE (Ovid), Scopus, and Web of Science. The search employed the keywords “compassion fatigue” and incorporated the timeframe March 1, 2020, to June 15, 2022, so that the most impactful waves of the COVID-19 pandemic were represented in the included literature, resulting in 1519, 2489, and 2246 studies, from the respective databases. These three databases were selected due to their likelihood of yielding results relevant to the research topic. To construct a comprehensive search strategy, a collection of keywords and index terms were identified from the titles and abstracts of relevant articles. The search strategy was further refined in collaboration with a social science librarian.

Structured search strategy

A systematic search was conducted across six scholarly electronic databases: MEDLINE (Ovid), Embase (Ovid), CINAHL, Scopus, and Web of Science. These databases were deliberately chosen to encompass a broad range of relevant findings within the current knowledge landscape regarding the research topic. The systematic search of the literature commenced once the scoping review was peer reviewed and revisions were addressed by the authors. Using the selected vocabulary and Boolean connectors as shown in Table 1 , a string of relevant search terms was developed. The search strategy was adapted accordingly for each individual database (e.g., Medical Subject Headings [MeSH] terms for MEDLINE [Ovid]). In the final stage of the search strategy, the reference lists of all included studies were manually examined to identify additional relevant studies.

Inclusion criteria

The inclusion criteria for this review was formulated using the PCC (Population, Concept, Context) mnemonic developed by JBI (Table 1 ). The participants included in this review were HCPs who were employed across healthcare systems during the COVID-19 pandemic (e.g., physicians, registered nurses, nurse practitioners, physician assistants, and licensed clinical social workers). The concept explored in this review focused on compassion fatigue among HCPs working in healthcare systems during the COVID-19 pandemic. The context of the study encompassed various care settings where HCPs carry out their professional activities across different clinical specialties (e.g., surgery, critical care, palliative care), as well as clinical settings (e.g., inpatient and outpatient). For the purposes of this scoping review, formal healthcare settings were broadly classified as those that provided health services and were situated within and administered by healthcare institutions.

This scoping review only included articles published in English. A time filter was applied to encompass studies conducted between 2020 to 2023, spanning the period from the onset of the COVID-19 pandemic to the present. A range of study designs were included in the review (i.e., experiments, quasi-experimental studies, analytical observational studies, descriptive observational studies, mixed-methods studies, and qualitative studies).

Exclusion criteria

Through the past two decades, compassion fatigue has been defined in different ways, sometimes being considered synonymous with burnout and secondary traumatic stress, or as an outcome resulting from both components [ 12 , 13 ]. Yet recently, it has been suggested that compassion fatigue is a focal concept related to the management of traumatic situations whereas burnout is a general concept that may have multiple contributors [ 26 ]. Due to the conceptual ambiguity surrounding compassion fatigue, articles that solely examine the components of compassion fatigue, such as burnout and secondary trauma, without directly addressing compassion fatigue itself, were excluded from consideration.

Studies that failed to meet the inclusion criteria or lacked full-text availability were excluded from the review. Additionally, editorials, letters to the editor, commentaries, and reviews were also excluded as they did not offer sufficient information for addressing the research questions.

Stage III: Study selection

After the full database searches were conducted, all identified citations were compiled and uploaded into Covidence. Any duplicate citations were automatically excluded.

Three reviewers (LH, CO, AG) independently screened the titles and abstracts of the identified studies to assess their eligibility according to the pre-established inclusion and exclusion criteria. Subsequently, the full texts of 736 selected studies were evaluated to arrive at the final list of articles for data extraction. The reasons for excluding specific studies were documented. Throughout the process, any disagreements that arose at each stage of study selection were resolved through discussions with a third reviewer (AG, SB).

The outcomes of the study selection process were presented in a flow diagram adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) guidelines (Fig.  1 ) [ 29 ]. Additionally, all the included studies underwent an assessment of their risk of bias (quality) using established critical appraisal tools from the Joanna Briggs Institute (JBI) for Evidence Synthesis [ 30 ]. Although not mandatory for scoping reviews, appraisals of study quality will contribute to the subsequent implications and future steps stemming from this scoping review [ 31 ]. The JBI provides critical appraisal checklists for various study designs, encompassing experimental, quasi-experimental, randomized controlled trials, observational, and qualitative study designs. One reviewer (CO) conducted the assessments of all the included studies, and a second reviewer (AG) verified the evaluations. Any discrepancies that arose were discussed and resolved in consultation with both reviewers. In line with the methodology of scoping reviews, no studies were excluded based on their quality assessments, ensuring a comprehensive understanding of the current state of the literature on compassion fatigue among HCPs during the COVID-19 pandemic. A summary of the quality assessments were presented in the results section of the review, while the full appraisals can be found in Additional file 1 .

PRISMA flow chart [ 28 ]

Stage IV: Data extraction

To facilitate data extraction aligned with the research objectives, a data-extraction template was developed by one reviewer (LH). This template encompassed various aspects of the included studies (i.e., authors, publication year, study populations, country, study design, aims, sample size, assessment instruments, risk factors, protective factors, consequences of compassion fatigue, and measures to prevent/manage/reduce compassion fatigue). Utilizing Covidence, two independent reviewers (LH, CO) extracted the relevant data from the studies included in the final list of citations.

Stage V: Risk of bias

Standardized tools developed by the Joanna Briggs Institute for respective study types were used to assess risk of bias (quality) for all studies included in the review [ 27 ]. The study appraisals were conducted by one reviewer (CO) and reviewed by another reviewer (AG). Any discrepancies were discussed and resolved together. While no studies were excluded based on the appraisal scores to ensure a comprehensive presentation of the available literature on compassion fatigue among healthcare providers, the findings for the risk of bias assessments are summarized in the results section and the full appraisals are presented in Additional file 1 .

Stage VI: Collating, summarizing, and reporting the results

To summarize and synthesize the findings, the study followed a three-step approach proposed by Levac et al. [ 32 ]: (1) collating and analyzing the collected data; (2) reporting the results and outcomes to address the study objectives; and (3) discussing the potential implications that findings hold for future research and policy considerations [ 31 ]. The review process adhered to the PRISMA Extension for Scoping Reviews checklist, which provided guidance for conducting the review and reporting the findings [ 26 ].

Search results

Figure  1 displays the PRISMA-ScR flowchart of the scoping review search strategy. The search and reference list initially yielded 11,715 studies. Of these, 5769 were excluded as duplicates. Following the title and abstract screening of the remaining studies, 5179 studies were excluded as they met the exclusion criteria. Finally, the full-texts of the remaining 736 studies were screened, and 712 were excluded as they did not meet the inclusion criteria. In total, 24 eligible studies were included in the review for further analysis.

Risk of bias of included studies

The complete assessment of risk of bias of all 24 included studies is available in Additional file 1 . Within the two mixed-methods studies risk of bias primarily stemmed from the quantitative strand of the studies with a lack of clarity provided about study inclusion criteria, study setting, and identification of confounding factors [ 29 ]. Other sources of bias in other quantitative studies were vagueness around the criteria used for outcome measurement [ 30 ] and only one study identified potential cofounding factors along with strategies to manage them [ 31 ]. Further shortcomings related to the failure to provide transparency around the use of valid and reliable outcome measures [ 23 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 ]. Within qualitative studies not all provided information about the researchers’ theoretical stance [ 29 , 41 , 43 ] and two studies did not provide documentation of ethics approval for the conducted research [ 43 , 44 ]. One included case report met most assessment criteria for risk of bias although more description of assessment, post-assessment condition and adverse events were warranted [ 45 ].

Characteristics of studies

Study characteristics are presented in Table 2 . Of the 24 eligible studies, 18 studies used quantitative methods [ 23 , 30 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 46 , 47 , 48 , 49 , 50 , 51 ], 3 studies used qualitative methods [ 43 , 44 , 45 ], and the remaining studies used mixed-methods approaches [ 29 , 41 , 52 ]. Additionally, 13 studies focused on the antecedents of compassion fatigue [ 23 , 29 , 33 , 34 , 35 , 36 , 40 , 41 , 42 , 45 , 46 , 47 , 48 ] and 5 studies examined the consequences of compassion fatigue [ 30 , 37 , 43 , 44 , 49 ]. Six studies were conducted in the United States, with the others being conducted in a range of countries including Ecuador, Spain, United Kingdom, Italy, Greece, Turkey, Iran, Uganda, Taiwan, Japan, Philippines, China, and India. These studies primarily focused on nurses, physicians, and other allied health professionals. The study samples included both male and female HCPs. Only one study focused exclusively on female HCPs [ 43 ].

A variety of assessment tools were used to measure compassion fatigue across included studies. Common tools included Compassion Fatigue Short Scale (CFSS) [ 33 , 47 , 48 ], Compassion Fatigue Scale (CFS) [ 30 , 49 ], Professional Quality of Life Scale Version 5 (ProQoL 5) [ 23 , 29 , 29 , 31 , 35 , 36 , 38 , 39 , 40 , 41 , 42 , 50 , 51 ], Work-Related Quality of Life Scale (WRQoL) [ 46 ], and Compassion Fatigue and Satisfaction Self-Test (CFST) [ 37 , 52 ] (Table 3 ).

The time period of the study period shows that most of the studies were conducted in the first six months of 2020, coinciding with the World Health Organization’s declaration of the COVID-19 outbreak as a pandemic [ 54 ]. No studies included in the review were conducted between March 2021 and May 2023 (Fig.  2 ).

The time trend of study periods on compassion fatigue in HCPs during the COVID-19 pandemic

Findings were synthesized and presented using the following 4 themes: (1) prevalence of compassion fatigue, (2) antecedents of compassion fatigue (individual-Level, organizational-Level, and systems-level factors), (3) consequences of compassion fatigue, and (4) interventions for compassion fatigue.

Theme 1: Prevalence of compassion fatigue

Of the studies reviewed, five measured the prevalence of compassion fatigue among HCPs during the COVID-19 pandemic [ 23 , 30 , 31 , 36 , 41 ]. In a study conducted in Spain, 306 out of 506 (60.4%) HCPs reported high levels of compassion fatigue while 170 (33.6%) showed moderate levels of compassion fatigue (ProQoL 5: M = 19.9, SD = 7.6) [ 36 ]. In a sample composed of 395 Ugandan frontline nurses, 49.11% of the nurses reported high levels of compassion fatigue, while 29.6% experienced moderate levels of compassion fatigue [ 23 ]. Over half of the nurses in the study (54.94%) reported direct exposure to COVID-19 cases. A study conducted in Greece found that in a sample of 105 nurses, the majority of nurses (51.4%) experienced moderate levels of compassion fatigue (ProQoL 5: M = 22.26, SD = 6.76) [ 41 ]. In a Taiwanese study of 503 HCPs, the majority of the participants (63.2%) experienced low levels of compassion fatigue (ProQoL 5: M = 20.9, SD = 7.6) [ 31 ]. Finally, in a Filipino sample composed of 270 frontline nurses, 61.4% of the nurses reported low levels of compassion fatigue (CFS: M = 2.213, SD = 0.979) [ 30 ].

Theme 2: Antecedents of compassion fatigue

Individual-level factors.

Age and sex were key factors associated with compassion fatigue among participant HCPs. Younger HCPs with less experience were more likely to experience mental health issues and conflicting feelings with regards to providing care to COVID-19 patients [ 23 , 29 , 44 , 46 ]. Seven studies included in the review determined that female HCPs were more likely than male HCPs to experience compassion fatigue [ 23 , 35 , 36 , 38 , 40 , 50 , 52 ]. Physicians were also reported to have higher levels of compassion fatigue compared to nurses in three studies [ 36 , 38 , 39 ]. While nursing assistants had higher levels of compassion fatigue when compared to nurses in one study (ProQol 5: Nursing assistants = 29.15 ± 6.94; Nurse = 25.68 ± 5.87) [ 29 ]. Furthermore, the risk was higher in permanent workers compared to temporary workers (ProQoL 5: Permanent = 2.48 ± 1.29; Temporary = 2.11 ± 1.15; P -value < 0.05) [ 35 ]. One included study determined that marital status and education levels were not correlated with compassion fatigue [ 23 ]. Psychiatric comorbidities such as past trauma, burnout, stress, anxiety, and depression exacerbated HCPs’ psychological well-being across a number of included studies [ 31 , 33 , 36 , 38 , 39 , 41 , 49 , 50 ]. Other psychological factors such as excessive empathetic engagement, sensitive sensory processes, and overidentification from frequent witnessing of patient suffering and deaths were found to aggravate the development of compassion fatigue [ 34 , 39 , 45 ]. The inability to cope with the rapidly evolving landscape of healthcare provision and a lack of self-care contributed to increased burden and blurring of role boundaries between professional and private lives [ 29 , 41 , 43 , 44 , 51 , 52 ]. One study that used Compassion Fatigue and Satisfaction Self-Tests and a questionnaire of personal and professional characteristics found that feelings of underappreciation, insufficient compensations, and social isolation incurred psychological burden on pediatric sub-specialists [ 52 ]. Additionally, a decrease in occupational hardiness, as measured by the Occupational Hardiness Questionnaire, increased the risk of compassion fatigue among HCPs in two studies [ 42 , 50 ]. Negative outcomes to the HCPs’ families and concerns revolving around their patients’ families also predicted higher risk of experiencing compassion fatigue [ 45 , 48 , 52 ]. Finally, HCPs’ fear of COVID-19 with regards to infection and transmission was identified as a predictor of compassion fatigue [ 29 , 40 , 43 , 44 , 47 ].

Two studies identified social support from family, friends, peers, and hospital leadership as a crucial protective factor for compassion fatigue [ 43 , 52 ]. Coping mechanisms such as venting and exercising were found to help alleviate stress among HCPs [ 44 ]. Psychological qualities such as compassion satisfaction, professional satisfaction, resilience, vigor, and hardiness were found to help protect the psychological health of HCPs as well as reducing turnover intention and increasing perceived quality of care [ 30 , 34 , 36 , 37 , 39 , 40 , 42 , 46 , 50 ]. Self-care, self-awareness of limitations, and self-regulation of emotions were crucial for reducing risk of compassion fatigue in two studies comprised of physicians and nurses [ 44 , 50 ]. Lastly, spirituality, religiosity, and meditation also served as protective factors in three studies on compassion fatigue in HCPs [ 41 , 44 , 51 ].

Organizational-level factors

In five of the articles reviewed, increased workload [ 23 , 29 , 44 , 45 ], long working hours [ 23 , 29 , 44 , 45 ], and increased number of patients [ 50 ] were identified as common predictors of compassion fatigue. Furthermore, providing direct care to COVID-19 patients, which were often emotionally challenging cases, exacerbated the psychological risks to HCPs [ 23 , 36 , 46 , 48 , 50 ]. Chronic exposure to a dynamic work environment also increased the risk of compassion fatigue among HCPs [ 29 ]. Lack of access to suitable PPEs and lack of foresight from management and human resources teams regarding infection control guidelines contributed to HCPs’ distress [ 29 ]. Adjusting to the discomfort caused by wearing PPEs presented as a challenge to maintaining the efficiency of work activities [ 29 ]. Lastly, in two studies, HCPs identified that while there were plenty of wellness resources provided by healthcare organizations to support mindfulness, there was a lack of practical and pragmatic resources for social and emotional support, work-life balance, and remuneration [ 23 , 43 ].

Positive work conditions, such as a visible presence and engagement by leadership and management, as well as a positive work culture allowing HCPs to seek help without fear of judgment was found to be important protective factors against the development of compassion fatigue [ 44 ]. The social aspects of teamwork facilitated the sharing of feelings of trauma which in turn contributed to resilience and improved psychological well-being among HCPs in three studies [ 41 , 43 , 44 ]. One study observed that workplace wellness activities and a sense of feeling valued can prevent high levels of compassion fatigue [ 52 ]. Words of appreciation from supervisors boosted morale for some HCPs [ 44 ]. Attention to workplace safety in the form of PPEs and early access to vaccines alleviated the fear of infection [ 44 ]. Finally, two studies determined that adequate preparation and education to handle COVID-19 cases and increased autonomy decreased the risk of compassion fatigue and increased professional fulfillment [ 42 , 44 ].

Systems-level factors

Significant and frequently changing public health measures over the course of the pandemic presented a challenge as they were disruptive to workflow and resulted in uncertainty, feelings of inadequacy, and distress among HCPs across a range of geographical contexts [ 29 , 41 , 43 , 49 ]. Increases in the incidence of COVID-19 cases also contributed to a rise in the number of hospital admissions, aggravating HCPs’ workload [ 35 ]. Social-distancing policies precluded informal team interactions, such as sharing meals together, which posed a risk to HCPs’ psychological well-being by decreasing social support [ 43 , 52 ]. Transitions to tele-health also increased social isolation [ 43 ]. A theme that emerged was the negative impact of stigma on HCPs, with their proximity to contagion, as a possible risk factor [ 35 , 41 ]. Aggressive behaviors and verbal abuse from patients were sources of emotional stress for some HCPs [ 44 ]. Finally, negative peer pressure was identified as a barrier to HCPs engaging in self-care as they felt pressure to conform to sociocultural norms of an expected level of dedication [ 44 ]. In contrast to the impacts of stigma, a positive perception of one’s own profession is related to increased commitment and decreased compassion fatigue [ 46 ].

Theme 3: Consequences of compassion fatigue

The findings of one study suggested that compassion fatigue associated with HCP’s professional practice impacted their private lives, predicting greater parental burnout ( r  = 0.542), child abuse ( r  = 0.468), child neglect ( r  = 0.493), spouse conflict ( r  = 0.340), and substance abuse ( r  = 0.298) [ 48 ]. This study identified factors such as direct care of COVID-19 patients ( r  = 0.255), exposure to patient death and suffering due to COVID-19 ( r  = 0.281), and family income loss due to COVID-19 ( r  = 0.366) as risk factors for compassion fatigue [ 48 ]. Additionally, at an organizational-level, two studies conducted in 2020 and 2021 observed that Turkish and Filipino HCPs who reported compassion fatigue also reported lower job satisfaction and reduced professional commitment [ 30 , 46 ]. Consequently, elevated compassion fatigue also increased organizational turnover intent among Filipino HCPs (β = 0.301, P -value = 0.001) [ 30 ]. A study conducted in China found that compassion fatigue predicted negative behavioral intentions towards treating COVID-19 patients, as measured by the Attitude, Subjective Norms, and Behavioral Intention of Nurses toward Mechanically Ventilated Patients (ASIMP) questionnaire [ 33 ]. This suggests that quality of care may be adversely impacted [ 33 ]. Finally, an American study observed that compassion fatigue among HCPs was associated with deteriorating workplace culture [ 52 ].

• Patient care

The provision of care during the pandemic was impacted by the general lack of preparation for handling novel tasks experienced by many HCPs [ 23 ]. Findings from one study found that many HCPs (73%) experienced a shift in their clinical practice setting, for example, from in-personal care to virtual telehealth consults as a result of the pandemic [ 43 ]. HCPs also experienced an increase in the need to provide palliative care as a result of the negative health impacts of COVID-19, something they may have had limited prior experience with [ 43 ]. In a case study conducted in Japan, the physician reported feeling inexperienced with handling the psychological impact of the pandemic experienced by not only the patients but also the patients’ family [ 45 ]. The consequences of not being able to provide optimal care was found to exacerbate feelings of guilt, powerlessness, and frustration in HCPs [ 41 , 43 ]. In turn, study findings suggest that worsening compassion fatigue may reduce the quality of care provided by HCPs because it has been found to be a significant predictor of negative behavioral intention [ 30 , 33 , 40 , 52 ].

Theme 4: Interventions for compassion fatigue

Two studies in Japan and Uganda investigated potential interventions to support HCPs experiencing COVID-19 related compassion fatigue. On an individual-level, regularly engaging in self-care activities such as expressions of gratitude as well as learning how to recognize signs and symptoms of compassion fatigue were identified as crucial first steps in its management [ 45 , 52 ]. Emotional support from colleagues and mental health specialists was found to be effective in improving the mental health of a Japanese physician experiencing compassion fatigue [ 45 ]. Findings of two studies identified the need for a systematic approach to monitor the progression of psychological symptoms and providing tailored resources in a timely manner to HCPs to help ameliorate compassion fatigue and its consequences [ 29 , 45 ]. Suggested strategies included: facilitating regular consultations with each department [ 45 , 52 ], increasing the staffing number of HCPs in busy departments [ 23 , 45 ], and providing PPEs and vaccines in a timely manner [ 23 , 52 ]. Lastly, findings from two studies in Uganda and the United States suggested that increased remuneration may prevent or minimize compassion fatigue [ 23 , 52 ].

Key findings

This scoping review sought to provide a comprehensive summary of the literature published between January 2020 and May 2023 on the impact of the COVID-19 pandemic on compassion fatigue among HCPs and its subsequent impact on patient care. Most of the included studies were conducted in 2020 and used cross-sectional study designs. Given that the COVID-19 outbreak was declared a global health emergency in early 2020 [ 1 ], cross-sectional study designs were well-placed to provide prompt and important insights on compassion fatigue across the HCP population. Review findings were presented using four themes addressing the prevalence, antecedents, consequences, and consequences of compassion fatigue in HCPs. The prevalence of compassion fatigue was observed to vary across countries. The negative psychological outcomes reported by included studies were precipitated by individual-level factors such as age and occupational role; organizational-factors such as lack of access to PPE; and systems-level factors such as loss of social engagement and stigma. The consequences of compassion fatigue impacted HCPs’ personal and professional roles. Findings suggest an urgent need for policy makers, health managers, and team leaders to develop and implement strategies that target the potential root causes of compassion fatigue in HCPs.

Prevalence of compassion fatigue

Among the five studies that measured prevalence of compassion fatigue, results were highly variable across countries [ 23 , 30 , 31 , 36 , 41 ]. This may be attributed to differences in preparedness for infection containment and variability among health systems’ preparation and ability to respond to supply chain issues [ 53 ]. Taiwan provides an example of how digital technologies were adopted to improve disease surveillance and monitor medical supply chains [ 55 ]. Using the stringent Identify-Isolate-Inform model in conjunction with public mask-wearing and physical distancing, the spread of the disease was effectively contained in Taiwan [ 53 ]. Consequently, despite not enforcing lockdowns, Taiwan blocked the first wave of cases and slowed down subsequent outbreaks, which may contribute to the observed low prevalence of compassion fatigue among HCPs [ 56 ]. In the Philippines, responses to disease outbreaks varied across different municipalities and provinces [ 57 ]. Effective containment measures such as strict border control and early lockdowns in addition to plentiful medical supplies and personnel allowed certain regions to mount a strong response to this public health emergency, subsequently resulting in the observed low prevalence of compassion fatigue among HCPs [ 57 ]. In Uganda, there were generally low levels of preparedness with regards to the infection identification, PPE supply, access to hand-washing facilities, and establishment of isolation facilities [ 58 ]. This may have contributed to an overwhelmed healthcare system and overworked HCPs as the surge of cases was exacerbated by the shortage of disease containment resources [ 58 ]. In April 2020, Spain experienced the second highest infection incidence in the world [ 59 ]. The Spanish health system was overwhelmed by the abundance of patients due to lack of HCPs [ 60 ], hospital capacity, and material supplies [ 59 ]. An increase in compassion fatigue among HCPs was also observed in recent studies from Italy and Canada [ 61 , 62 ]. Overall, the various strategies used to address the resultant COVID-19-related public health crisis presented distinctive challenges to HCPs in different countries. Caution must be taken when interpreting the study findings given the contextual differences across various healthcare systems. The psychological burden and prevalence of compassion fatigue subsequently varied depending on the context.

Antecedents of compassion fatigue

The findings of this review suggest that individual characteristics such as age and occupational role are significant contributing factors to the development of compassion fatigue during COVID-19 [ 63 ]. Specifically, older HCPs were less likely to experience compassion fatigue than younger HCPs according to regression analyses [ 23 , 29 , 44 , 46 ]. This observation may be attributed to their increased work experience. Resilience was also positively linearly related to age [ 64 ]. Factors identified as potential contributors to the observed age-related advantage in wellbeing were access to job resources, better job security, work-life balance, and coping skills [ 64 ]. The compounding of stressors such as an increase in workload during the COVID-19 pandemic could have exacerbated the psychological health of younger HCPs. In the context of telework, older employees tended to create clear boundaries between work and non-work responsibilities [ 64 ]. The rise in telework among HCPs was mostly a consequence of the COVID-19 pandemic which may have increased the psychological burden on younger HCPs [ 65 ]. In addition, a study examining demographic predictors of resilience in nurses reported that younger nurses had less exposure to stress, and thus have fewer opportunities to develop skills in stress management [ 66 ]. As a result of these factors, the younger HCPs were at high risk for compassion fatigue during the COVID-19 pandemic. Interestingly, three of the included studies in this review also observed that physicians were at a higher risk of compassion fatigue compared to nurses [ 36 , 38 , 39 ]. This difference may be attributed to the burden of responsibility in relation to breaking bad news, a task that is often the physicians’ responsibility [ 67 ]. A study examining compassion fatigue in HCPs determined that conflict arising during patient interactions placed HCPs at a risk for compassion fatigue [ 68 ]. Delivery of bad or uncertain news also predicted a greater mental health burden in HCPs [ 68 ].

At the organizational level, findings from the studies included in this review identified that a lack of access to PPE was a contributor to compassion fatigue in HCPs during COVID-19 [ 29 , 52 ]. Specifically, one study reported that the fear of infection and transmission to patients, family, and friends added to the concern of HCPs working in high-risk environments [ 69 ]. This finding can potentially be explained by the increased vulnerability that HCPs experience following a lag in the provision of PPE. Several organizational factors were determined as potential barriers to the distribution of PPE; the unprecedented nature of the pandemic presented challenges for maintaining domestic inventories [ 70 ]. Disruptions to the PPE global supply chain also amplified the equipment shortage [ 70 ]. This finding highlights the importance of monitoring and ensuring that domestic health supplies are adequately stocked.

At the system level, loss of social engagement [ 43 , 52 ] and stigma [ 35 , 41 ] were identified in the studies included in the review as antecedents to compassion fatigue. Public policies such as social-distancing and occupancy capacity limits negatively impact social interactions which may explain the loss of social engagement in addition to worsening mental health well-being in HCPs [ 71 ]. As certain practices transition to telehealth, other studies have found increased mental fatigue and difficulty with maintaining empathetic rapport, which has important implications on patient care [ 72 , 73 ]. In addition, other studies have found that given the proximity of their role to contagion, stigma towards HCPs from patients increased during COVID-19 [ 74 , 75 ]. Consequently, the combinatorial experience of being socially isolated and stigmatized may worsen mental health outcomes [ 76 ]. This points to a need for increased access to support services for HCPs such as virtual communities.

Consequences of compassion fatigue

Review findings suggest that compassion fatigue impacted the private and professional lives of HCPs. The risk for parental burnout has increased across many occupations during the pandemic [ 77 ]. Factors related to low levels of social support, lack of leisure time, and greater parental responsibilities in face of education disruptions adds to the psychological burden of parents [ 77 ]. HCPs were placed in a unique position having to work in highly stressful environments while also balancing household responsibilities and increased challenges related to childcare [ 48 , 78 ]. This finding highlights a need for the provision of child support services for HCPs or a reduction in workload to alleviate the burden of parental and homecare responsibilities particularly in times of public health crises.

Beyond their private lives, this review has found that decreases in HCPs’ professional commitment due to compassion fatigue, may endanger the quality of patient care delivered [ 79 ]. In particular, this may be attributed to the surge in palliative care cases during the pandemic in conjunction with an unprepared workforce, creating psychological stress for HCPs [ 80 ]. In a study examining palliative care preparedness during the pandemic, a lack of core palliative care training and expertise among frontline HCPs [ 81 ] meant many felt emotionally unprepared to address cases with seriously ill patients [ 45 ]. An increased frequency of breaking bad news to patients’ families was associated with negative psychological outcomes [ 82 ]. Providing training on relevant communication skills may protect HCPs from compassion fatigue [ 83 , 84 ].

Implications

The findings of this review highlight the urgency to provide support for HCPs who may be at risk for compassion fatigue which could have subsequent impacts on the provision of patient care [ 85 ]. To address the antecedents of compassion fatigue, this scoping review has identified a need for increased staffing, recruitment, and retention efforts on the part of hospital human resources departments [ 23 , 45 ]. Interventions suggested by studies included in the review encompass the monitoring of psychological well-being among HCPs to inform timely provision of resources [ 29 , 45 ]. Specifically, structured debriefing, training on self-care routine, reduced workload, and normalization of trauma-related therapy are essential interventions [ 86 ]. Additionally, a study identified that fostering collaborative workplace culture encourages social and emotional support among staff [ 45 ]. Certain hospitals have adopted “wobble rooms” as a private unwinding and venting space for employees [ 87 ]. Studies have observed that interventions aimed at improving the well-being of HCPs resulted in enhanced quality and safety of care being delivered [ 75 ].

Strengths and limitations

There are both strengths and limitations in this review. Although some literature reviews focused on the psychological health status of HCPs (e.g., burnout, anxiety, depression), very few studies have specifically explored compassion fatigue. Reviews that considered the impact of the COVID-19 pandemic on HCPs were even more limited. It is known that compassion is a cornerstone of quality health care improvement and increases successful medical outcomes [ 88 , 89 , 90 ]. Nevertheless, prolonged exposure to distressing events by HCPs, such as patient death and suffering, results in the absorption of negative emotional responses and leads to the development of compassion fatigue [ 91 ]. This scoping review presents an extensive exploration of the current body of literature on compassion fatigue among HCPs during the COVID-19 pandemic. Another strength in this study lies in the transparency and reproducibility of the methodology. The scoping review protocol has been published in a peer-reviewed journal to establish high methodological standards for the final scoping review [ 92 ]. Additionally, the study plan was pre-registered with Open Science Framework to ensure commitment to the methodology. Double extraction was performed to ensure that a comprehensive descriptive summary of the studies was achieved.

Some limitations include the short time frame chosen for the included studies that were published since the COVID-19, which may have constrained the breadth and quality of the studies. Longitudinal studies may not be captured in the review as this study methodology requires a prolonged period of time to yield meaningful observations. More data is needed to support conclusions on the impact of compassion fatigue on patient care. Additionally, none of the studies included in the review were conducted between March 2021 and May 2023, which may miss out on meaningful trends in levels of compassion fatigue in HCPs. This scoping review only included literature published in English so studies published in other languages were not assessed. Additionally, no comparisons of compassion fatigue were made among the HCP groups in spite of potentially relevant differences such as patient exposure. There was also a lack of allied health profession representation, with the majority of the study population being nurses or physicians. Lastly, grey literature was not included in this scoping review which may delimitate the information included in the scoping review.

There were recurring themes related to limitations in the included research studies. Several studies identified sampling issues including small sample sizes, restricted sample frame, low response rate, and selection error [ 23 , 29 , 31 , 38 , 39 , 40 , 41 , 42 , 43 , 47 , 50 , 51 , 83 ]. Other studies have called for investigations into how different sociodemographic factors, other psychiatric diseases, health care settings, and workplace environment impact compassion fatigue in HCPs [ 38 , 39 , 47 , 48 , 83 ]. One study observed a lack of homogeneity in the sample due to an overrepresentation of female HCPs in the sample [ 38 ]. Lastly, many studies employed a cross-sectional study design which limits the interpretation of the data in terms of causality [ 23 , 30 , 31 , 34 , 42 , 47 , 48 , 50 ]. While there are limitations to the study, a comprehensive summary of existing literature may be useful to inform future research and policies.

Future research is needed to examine the longitudinal impacts of COVID-19 on compassion fatigue in HCPs. Moreover, research in this area could be strengthened by including a consultation phase with external experts on compassion fatigue to improve the robustness of the scoping review.

Conclusions

The COVID-19 pandemic presented a unique set of challenges to healthcare systems across the globe. This scoping review indicated that the prevalence of compassion fatigue was inconsistent across countries and may reflect the variability of pandemic preparedness among the individual countries. Primary risk factors for the development of compassion fatigue included being younger, female, a physician or nurse, and having limited access to PPE in conjunction with an excessive workload and prolonged work hours. The negative impacts of compassion fatigue were experienced at the individual and organizational level. The findings suggest there is a systemic need to assess, monitor and support health professionals’ well-being particularly during conditions of protracted health crises such as a pandemic. In addition, many health systems and sectors are facing a profound health human resources crisis and therefore ongoing efforts must be made to improve workplace environments and increase recruitment and retention efforts. Lastly, pandemic planning must include provisions to support health providers’ ability to safely do their jobs while also minimizing negative impacts to their health and well-being.

Availability of data and materials

All the material presented in the manuscript is owned by the authors and/or no permissions are required.

Coronavirus disease (COVID-19) pandemic. Accessed 16 Jan 2023. https://www.who.int/europe/emergencies/situations/covid-19

Wilder-Smith A, Osman S. Public health emergencies of international concern: a historic overview. J Travel Med. 2020;27(8):taaa227. https://doi.org/10.1093/jtm/taaa227 .

Article   PubMed   Google Scholar  

Gristina GR, Piccinni M. COVID-19 pandemic in ICU. limited resources for many patients: approaches and criteria for triaging. Minerva Anestesiol. 2021;87(12):1367–79. https://doi.org/10.23736/S0375-9393.21.15736-0 .

Chaka EE, Mekuria M, Melesie G. Access to Essential personal safety, availability of personal protective equipment and perception of healthcare workers during the COVID-19 in public hospital in West Shoa. Infect Drug Resist. 2022;15:2315–23. https://doi.org/10.2147/IDR.S344763 .

Article   PubMed   PubMed Central   Google Scholar  

Gholami M, Fawad I, Shadan S, et al. COVID-19 and healthcare workers: a systematic review and meta-analysis. Int J Infect Dis. 2021;104:335–46. https://doi.org/10.1016/j.ijid.2021.01.013 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Schug C, Geiser F, Hiebel N, et al. Sick Leave and Intention to Quit the Job among Nursing Staff in German Hospitals during the COVID-19 Pandemic. Int J Environ Res Public Health. 2022;19(4):1947. https://doi.org/10.3390/ijerph19041947 .

Lancet T. COVID-19: protecting health-care workers. Lancet Lond Engl. 2020;395(10228):922. https://doi.org/10.1016/S0140-6736(20)30644-9 .

Article   Google Scholar  

Beck E, Daniels J. Intolerance of uncertainty, fear of contamination and perceived social support as predictors of psychological distress in NHS healthcare workers during the COVID-19 pandemic. Psychol Health Med. Published online July 6, 2022:1–13. https://doi.org/10.1080/13548506.2022.2092762.

Nikeghbal K, Kouhnavard B, Shabani A, Zamanian Z. Covid-19 effects on the mental workload and quality of work life in Iranian nurses. Ann Glob Health. 2021;87(1):79. https://doi.org/10.5334/aogh.3386.

WHO Coronavirus (COVID-19) Dashboard. Accessed 16 Jan 2023. https://covid19.who.int

Lluch C, Galiana L, Doménech P, Sansó N. The Impact of the COVID-19 pandemic on burnout, compassion fatigue, and compassion satisfaction in healthcare personnel: a systematic review of the literature published during the first year of the pandemic. Healthcare. 2022;10(2):364. https://doi.org/10.3390/healthcare10020364 .

Salmond E, Salmond S, Ames M, Kamienski M, Holly C. Experiences of compassion fatigue in direct care nurses: a qualitative systematic review. JBI Evid Synth. 2019;17(5):682. https://doi.org/10.11124/JBISRIR-2017-003818 .

Sinclair S, Raffin-Bouchal S, Venturato L, Mijovic-Kondejewski J, Smith-MacDonald L. Compassion fatigue: A meta-narrative review of the healthcare literature. Int J Nurs Stud. 2017;69:9–24. https://doi.org/10.1016/j.ijnurstu.2017.01.003 .

Majid U, Hussain SAS, Zahid A, Haider MH, Arora R. Mental health outcomes in health care providers during the COVID-19 pandemic: an umbrella review. Health Promot Int. 2023;38(2):daad025. https://doi.org/10.1093/heapro/daad025 .

Rahmani F, Hosseinzadeh M, Gholizadeh L. Complicated grief and related factors among nursing staff during the Covid-19 pandemic: a cross-sectional study. BMC Psychiatry. 2023;23(1):73. https://doi.org/10.1186/s12888-023-04562-w .

Statement on the fifteenth meeting of the IHR (2005) Emergency Committee on the COVID-19 pandemic. Accessed July 6, 2023. https://www.who.int/news/item/05-05-2023-statement-on-the-fifteenth-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-coronavirus-disease-(covid-19)-pandemic

Ghahramani S, Kasraei H, Hayati R, Tabrizi R, Marzaleh MA. Health care workers’ mental health in the face of COVID-19: a systematic review and meta-analysis. Int J Psychiatry Clin Pract. 2022;0(0):1–10. https://doi.org/10.1080/13651501.2022.2101927 .

Article   CAS   Google Scholar  

Nie A, Su X, Zhang S, Guan W, Li J. Psychological impact of COVID-19 outbreak on frontline nurses: a cross-sectional survey study. J Clin Nurs. 2020;29(21–22):4217–26. https://doi.org/10.1111/jocn.15454 .

Jalili M, Niroomand M, Hadavand F, Zeinali K, Fotouhi A. Burnout among healthcare professionals during COVID-19 pandemic: a cross-sectional study. Int Arch Occup Environ Health. 2021;94(6):1345–52. https://doi.org/10.1007/s00420-021-01695-x .

Iddrisu M, Poku CA, Mensah E, Attafuah PYA, Dzansi G, Adjorlolo S. Work-related psychosocial challenges and coping strategies among nursing workforce during the COVID-19 pandemic: a scoping review. BMC Nurs. 2023;22(1):210. https://doi.org/10.1186/s12912-023-01368-9 .

Yang BJ, Yen CW, Lin SJ, et al. Emergency nurses’ burnout levels as the mediator of the relationship between stress and posttraumatic stress disorder symptoms during COVID-19 pandemic. J Adv Nurs. 2022;78(9):2861–71. https://doi.org/10.1111/jan.15214 .

Fukushima H, Imai H, Miyakoshi C, Naito A, Otani K, Matsuishi K. The sustained psychological impact of coronavirus disease 2019 pandemic on hospital workers 2 years after the outbreak: a repeated cross-sectional study in Kobe. BMC Psychiatry. 2023;23(1):313. https://doi.org/10.1186/s12888-023-04788-8 .

Amir K, Okalo P. Frontline nurses’ compassion fatigue and associated predictive factors during the second wave of COVID-19 in Kampala. Uganda Nurs Open. 2022;9(5):2390–6. https://doi.org/10.1002/nop2.1253 .

Calkins K, Guttormson J, McAndrew NS, et al. The early impact of COVID-19 on intensive care nurses’ personal and professional well-being: a qualitative study. Intensive Crit Care Nurs. 2023;76:103388. https://doi.org/10.1016/j.iccn.2023.103388 .

Sexton JB, Adair KC, Proulx J, et al. Emotional exhaustion among US health care workers before and during the COVID-19 Pandemic, 2019–2021. JAMA Netw Open. 2022;5(9):e2232748. https://doi.org/10.1001/jamanetworkopen.2022.32748 .

Slatten LA, David Carson K, Carson PP. Compassion fatigue and burnout what managers should know. Health Care Manag. 2011;30(4):325–33. https://doi.org/10.1097/HCM.0b013e31823511f7 .

Martin J. © Joanna Briggs Institute 2017 Critical Appraisal Checklist for Systematic Reviews and Research Syntheses. Published online 2017.

PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation | Annals of Internal Medicine. Accessed 16 Jan 16 2023. https://www.acpjournals.org/doi/full/ https://doi.org/10.7326/M18-0850?rfr_dat=cr_pub++0pubmed&url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org

Moreno-Mulet C, Sansó N, Carrero-Planells A, et al. The Impact of the COVID-19 Pandemic on ICU healthcare professionals: a mixed methods study. Int J Environ Res Public Health. 2021;18(17):9243. https://doi.org/10.3390/ijerph18179243 .

Labrague LJ, de los Santos JAA. Resilience as a mediator between compassion fatigue, nurses’ work outcomes, and quality of care during the COVID-19 pandemic. Appl Nurs Res. 2021;61:151476.

Su PA, Lo MC, Wang CL, et al. The correlation between professional quality of life and mental health outcomes among hospital personnel during the Covid-19 pandemic in Taiwan. J Multidiscip Healthc. 2021;14:3485–95. https://doi.org/10.2147/JMDH.S330533 .

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implementation Sci. 2010;5:69. https://doi.org/10.1186/1748-5908-5-69 .

Cheng J, Cui J, Yu W, Kang H, Tian Y, Jiang X. Factors influencing nurses’ behavioral intention toward caring for COVID-19 patients on mechanical ventilation: a cross-sectional study. PLoS ONE. 2021;16(11):e0259658. https://doi.org/10.1371/journal.pone.0259658 .

Pérez-Chacón M, Chacón A, Borda-Mas M, Avargues-Navarro ML. Sensory processing sensitivity and compassion satisfaction as risk/protective factors from burnout and compassion fatigue in healthcare and education professionals. Int J Environ Res Public Health. 2021;18(2):611. https://doi.org/10.3390/ijerph18020611 .

Ramaci T, Barattucci M, Ledda C, Rapisarda V. Social Stigma during COVID-19 and its Impact on HCWs Outcomes. Sustainability. 2020;12(9):3834. https://doi.org/10.3390/su12093834 .

Ruiz-Fernández MD, Ramos-Pichardo JD, Ibáñez-Masero O, Cabrera-Troya J, Carmona-Rega MI, Ortega-Galán ÁM. Compassion fatigue, burnout, compassion satisfaction and perceived stress in healthcare professionals during the COVID-19 health crisis in Spain. J Clin Nurs. 2020;29(21–22):4321–30. https://doi.org/10.1111/jocn.15469 .

Kase SM, Gribben JL, Guttmann KF, Waldman ED, Weintraub AS. Compassion fatigue, burnout, and compassion satisfaction in pediatric subspecialists during the SARS-CoV-2 pandemic. Pediatr Res. 2022;91(1):143–8. https://doi.org/10.1038/s41390-021-01635-y .

Article   CAS   PubMed   Google Scholar  

Carmassi C, Dell’Oste V, Bertelloni CA, et al. Gender and occupational role differences in work-related post-traumatic stress symptoms, burnout and global functioning in emergency healthcare workers. Intensive Crit Care Nurs. 2022;69:103154. https://doi.org/10.1016/j.iccn.2021.103154 .

Ruiz-Fernández MD, Ramos-Pichardo JD, Ibáñez-Masero O, Carmona-Rega MI, Sánchez-Ruiz MJ, Ortega-Galán ÁM. Professional quality of life, self-compassion, resilience, and empathy in healthcare professionals during COVID-19 crisis in Spain. Res Nurs Health. 2021;44(4):620–32. https://doi.org/10.1002/nur.22158 .

Yılmaz A, Bay F, Erdem Ö, Özkalp B. The professional quality of life for healthcare workers during the COVID-19 Pandemic in Turkey and the influencing factors. Bezmialem Sci. 2022;10(3):361–9. https://doi.org/10.14235/bas.galenos.2021.5837 .

Missouridou E, Mangoulia P, Pavlou V, et al. Wounded healers during the COVID-19 syndemic: Compassion fatigue and compassion satisfaction among nursing care providers in Greece. Perspect Psychiatr Care. 2022;58(4):1421–32. https://doi.org/10.1111/ppc.12946 . Published online September 10, 2021.

Zakeri MA, Rahiminezhad E, Salehi F, Ganjeh H, Dehghan M. compassion satisfaction, compassion fatigue and hardiness among nurses: a comparison before and during the COVID-19 Outbreak. Front Psychol. 2022;12:815180. https://doi.org/10.3389/fpsyg.2021.815180 .

Austin EJ, Blacker A, Kalia I. “Watching the tsunami come”: a case study of female healthcare provider experiences during the COVID-19 pandemic. Appl Psychol Health Well-Being. 2021;13(4):781–97. https://doi.org/10.1111/aphw.12269 .

Kong KYC, Ganapathy S. Are we in control of our demons?: understanding compassion satisfaction, compassion fatigue and burnout in an asian pediatric emergency department in a pandemic. Pediatr Emerg Care. 2022;38(3):e1058. https://doi.org/10.1097/PEC.0000000000002656 .

Nishihara T, Ohashi A, Nakashima Y, Yamashita T, Hiyama K, Kuroiwa M. Compassion fatigue in a health care worker treating COVID-19 patients: a case report. Biopsychosoc Med. 2022;16(1):10. https://doi.org/10.1186/s13030-022-00239-0 .

Kaya ŞD, Mehmet N, Şafak K. Professional commitment, satisfaction and quality of life of nurses during the COVID-19 Pandemic in Konya. Turkey Ethiop J Health Sci. 2022;32(2):393–404. https://doi.org/10.4314/ejhs.v32i2.20 .

Kottoor AS, Chacko N. Role of entrapment in relation between fear of Covid-19 and compassion fatigue among nurses. Int J Behav Sci. 2022;15(4):250–5. https://doi.org/10.30491/ijbs.2022.288846.1573 .

Stevenson MC, Schaefer CT, Ravipati VM. COVID-19 patient care predicts nurses’ parental burnout and child abuse: Mediating effects of compassion fatigue. Child Abuse Negl. 2022;130:105458. https://doi.org/10.1016/j.chiabu.2021.105458 .

Hochwarter W, Jordan S, Kiewitz C, et al. Losing compassion for patients? The implications of COVID-19 on compassion fatigue and event-related post-traumatic stress disorder in nurses. J Manag Psychol. 2022;37(3):206–23. https://doi.org/10.1108/JMP-01-2021-0037 .

Cuartero-Castañer ME, Hidalgo-Andrade P, Cañas-Lerma AJ. professional quality of life, engagement, and self-care in healthcare professionals in Ecuador during the COVID-19 Pandemic. Healthcare. 2021;9(5):515. https://doi.org/10.3390/healthcare9050515 .

Spiridigliozzi S. Exploring the relationship between faith and the experience of burnout, compassion fatigue, and compassion satisfaction for hospice workers during a global pandemic: a multidisciplinary study. Dr Diss Proj. Published online April 1, 2022. https://digitalcommons.liberty.edu/doctoral/3572

Gribben JL, Kase SM, Guttmann KF, Waldman ED, Weintraub AS. Impact of the SARS-CoV-2 pandemic on pediatric subspecialists’ well-being and perception of workplace value. Pediatr Res. Published online January 20, 2023:1–7. https://doi.org/10.1038/s41390-023-02474-9.

Chien LC, Beÿ CK, Koenig KL. Taiwan’s Successful COVID-19 mitigation and containment strategy: achieving quasi population immunity. Disaster Med Public Health Prep.:1–4. https://doi.org/10.1017/dmp.2020.357.

Cucinotta D, Vanelli M. WHO Declares COVID-19 a Pandemic. Acta Bio-Medica Atenei Parm. 2020;91(1):157–60. https://doi.org/10.23750/abm.v91i1.9397 .

Kuo S, Ou HT, Wang CJ. Managing medication supply chains: Lessons learned from Taiwan during the COVID-19 pandemic and preparedness planning for the future. J Am Pharm Assoc. 2021;61(1):e12–5. https://doi.org/10.1016/j.japh.2020.08.029 .

Cheng HY, Liu DP. Early Prompt Response to COVID-19 in Taiwan: Comprehensive surveillance, decisive border control, and information technology support. J Formos Med Assoc. Published online November 11, 2022. https://doi.org/10.1016/j.jfma.2022.11.002.

S. Talabis DA, Babierra AL, H. Buhat CA, Lutero DS, Quindala KM, Rabajante JF. Local government responses for COVID-19 management in the Philippines. BMC Public Health. 2021;21:1711. https://doi.org/10.1186/s12889-021-11746-0 .

Rashid N, Nazziwa A, Nanyeenya N, Madinah N, Lwere K. Preparedness, identification and care of COVID-19 cases by front line health workers in selected health facilities in mbale district uganda: a cross-sectional study. East Afr Health Res J. 2021;5(2):144–50. https://doi.org/10.24248/eahrj.v5i2.665 .

Alfonso Viguria U, Casamitjana N. Early Interventions and Impact of COVID-19 in Spain. Int J Environ Res Public Health. 2021;18(8):4026. https://doi.org/10.3390/ijerph18084026 .

Rodríguez-Almagro J, Hernández-Martínez A, Romero-Blanco C, Martínez-Arce A, Prado-Laguna MD, García-Sanchez FJ. Experiences and Perceptions of Nursing Students during the COVID-19 Crisis in Spain. Int J Environ Res Public Health. 2021;18(19):10459.

Dodek PM, Cheung EO, Burns KEA, et al. Moral distress and other wellness measures in Canadian critical care physicians. Ann Am Thorac Soc. 2021;18(8):1343–51. https://doi.org/10.1513/AnnalsATS.202009-1118OC .

Franza F, Basta R, Pellegrino F, Solomita B, Fasano V. The role of fatigue of compassion, burnout and hopelessness in healthcare: experience in the time of covid-19 outbreak. Psychiatr Danub. 32.

Coşkun Şimşek D, Günay U. Experiences of nurses who have children when caring for COVID-19 patients. Int Nurs Rev. 2021;68(2):219–27. https://doi.org/10.1111/inr.12651 .

Scheibe S, De Bloom J, Modderman T. Resilience during crisis and the role of age: involuntary telework during the COVID-19 Pandemic. Int J Environ Res Public Health. 2022;19(3):1762. https://doi.org/10.3390/ijerph19031762 .

Mann DM, Chen J, Chunara R, Testa PA, Nov O. COVID-19 transforms health care through telemedicine: Evidence from the field. J Am Med Inform Assoc JAMIA. 2020;27(7):1132–5. https://doi.org/10.1093/jamia/ocaa072 .

Afshari D, Nourollahi-darabad M, Chinisaz N. Demographic predictors of resilience among nurses during the COVID-19 pandemic. Work. 2021;68(2):297–303. https://doi.org/10.3233/WOR-203376 .

Monden KR, Gentry L, Cox TR. Delivering bad news to patients. Proc Bayl Univ Med Cent. 2016;29(1):101–2.

Sorenson C, Bolick B, Wright K, Hamilton R. Understanding compassion fatigue in healthcare providers: a review of current literature. J Nurs Scholarsh. 2016;48(5):456–65. https://doi.org/10.1111/jnu.12229 .

Alharbi J, Jackson D, Usher K. The potential for COVID-19 to contribute to compassion fatigue in critical care nurses. J Clin Nurs. 2020;29(15–16):2762–4. https://doi.org/10.1111/jocn.15314 .

Cohen J, van der Meulen Rodgers Y. Contributing factors to personal protective equipment shortages during the COVID-19 pandemic. Prev Med. 2020;141:106263.

Institute of Professional Psychology, Bahria University Karachi Campus, Karachi, Pakistan, Waris Nawaz M, Imtiaz S, Quaid-i-Azam University Islamabad, Islamabad, Pakistan, Kausar E, Institute of Professional Psychology, Bahria University Karachi Campus, Karachi, Pakistan. self-care of frontline health care workers: during covid-19 pandemic. Psychiatr Danub. 2020;32(3–4):557–62. https://doi.org/10.24869/psyd.2020.557 .

Mano MS, Morgan G. Telehealth, social media, patient empowerment, and physician burnout: seeking middle ground. Am Soc Clin Oncol Educ Book Am Soc Clin Oncol Annu Meet. 2022;42:1–10. https://doi.org/10.1200/EDBK_100030 .

Myronuk L. Effect of telemedicine via videoconference on provider fatigue and empathy: Implications for the Quadruple Aim. Healthc Manage Forum. 2022;35(3):174–8. https://doi.org/10.1177/08404704211059944 .

Abuhammad S, Alzoubi KH, Al‐Azzam S, et al. Stigma toward healthcare providers from patients during COVID‐19 era in Jordan. Public Health Nurs Boston Mass. Published online March 25, 2022: https://doi.org/10.1111/phn.13071

Nashwan AJ, Valdez GFD, AL-Fayyadh S, et al. Stigma towards health care providers taking care of COVID-19 patients: a multi-country study. Heliyon. 2022;8(4):e09300. https://doi.org/10.1016/j.heliyon.2022.e09300 .

Shiu C, Chen WT, Hung CC, Huang EPC, Lee TSH. COVID-19 stigma associates with burnout among healthcare providers: evidence from Taiwanese physicians and nurses. J Formos Med Assoc Taiwan Yi Zhi. 2022;121(8):1384–91. https://doi.org/10.1016/j.jfma.2021.09.022 .

Griffith AK. Parental burnout and child maltreatment during the COVID-19 Pandemic. J Fam Violence. 2022;37(5):725–31. https://doi.org/10.1007/s10896-020-00172-2 .

Çakmak G, Öztürk ZA. Being both a parent and a healthcare worker in the pandemic: who could be exhausted more? Healthcare. 2021;9(5):564. https://doi.org/10.3390/healthcare9050564 .

Cavanagh N, Cockett G, Heinrich C, et al. Compassion fatigue in healthcare providers: a systematic review and meta-analysis. Nurs Ethics. 2020;27(3):639–65. https://doi.org/10.1177/0969733019889400 .

Boufkhed S, Harding R, Kutluk T, Husseini A, Pourghazian N, Shamieh O. What is the preparedness and capacity of palliative care services in Middle-Eastern and North African Countries to Respond to COVID-19? a rapid survey. J Pain Symptom Manage. 2021;61(2):e13–50. https://doi.org/10.1016/j.jpainsymman.2020.10.025 .

Gelfman LP, Morrison RS, Moreno J, Chai E. Palliative care as essential to a hospital system’s pandemic preparedness planning: how to get ready for the next wave. J Palliat Med. 2021;24(5):656–8. https://doi.org/10.1089/jpm.2020.0670 .

Messerotti A, Banchelli F, Ferrari S, et al. Investigating the association between physicians self-efficacy regarding communication skills and risk of “burnout.” Health Qual Life Outcomes. 2020;18:271. https://doi.org/10.1186/s12955-020-01504-y .

Gribben JL, Kase SM, Waldman ED, Weintraub AS. A cross-sectional analysis of compassion fatigue, burnout, and compassion satisfaction in pediatric critical care physicians in the United States. Pediatr Crit Care Med J Soc Crit Care Med World Fed Pediatr Intensive Crit Care Soc. 2019;20(3):213–22. https://doi.org/10.1097/PCC.0000000000001803 .

Sengupta M, Roy A, Gupta S, Chakrabarti S, Mukhopadhyay I. Art of breaking bad news: a qualitative study in Indian healthcare perspective. Indian J Psychiatry. 2022;64(1):25–37. https://doi.org/10.4103/indianjpsychiatry.indianjpsychiatry_346_21 .

Cross LA. Compassion fatigue in palliative care nursing: a concept analysis. J Hosp Palliat Nurs. 2019;21(1):21. https://doi.org/10.1097/NJH.0000000000000477 .

Paiva-Salisbury ML, Schwanz KA. Building compassion fatigue resilience: awareness, prevention, and intervention for pre-professionals and current practitioners. J Health Serv Psychol. 2022;48(1):39–46. https://doi.org/10.1007/s42843-022-00054-9 .

Jun 8, information 2020 | For more, Corpuz-Bosshart contact L. ‘Wobble room’ provides time-out for COVID-19 frontliners. UBC News. Published June 8, 2020. Accessed 17 Jan 2023. https://news.ubc.ca/2020/06/08/making-a-difference-wobble-room-provides-time-out-for-covid-19-frontliners/

Gupta N, Dhamija S, Patil J, Chaudhari B. Impact of COVID-19 pandemic on healthcare workers. Ind Psychiatry J. 2021;30(Suppl 1):S282–4. https://doi.org/10.4103/0972-6748.328830 .

Menon GR, Yadav J, Aggarwal S, et al. Psychological distress and burnout among healthcare worker during COVID-19 pandemic in India—a cross-sectional study. PLoS ONE. 2022;17(3):e0264956. https://doi.org/10.1371/journal.pone.0264956 .

Nishimura Y, Miyoshi T, Sato A, et al. Burnout of healthcare workers amid the COVID-19 Pandemic: a follow-up study. Int J Environ Res Public Health. 2021;18(21):11581. https://doi.org/10.3390/ijerph182111581 .

Jemal K, Hailu D, Mekonnen M, Tesfa B, Bekele K, Kinati T. The importance of compassion and respectful care for the health workforce: a mixed-methods study. J Public Health. 2023;31(2):167–78. https://doi.org/10.1007/s10389-021-01495-0 .

Hui L, Garnett A, Oleyniov C, Boamah S. Compassion fatigue in health providers during the COVID-19 pandemic: A scoping review protocol. BMJ Open. 2023;13:e069843. https://doi.org/10.1136/bmjopen-2022-069843 .

Download references

Acknowledgements

I declare that the authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and/or discussion reported in this paper.

Author information

Authors and affiliations.

Arthur Labatt Family School of Nursing, Western University, London, ON, Canada

Anna Garnett & Christina Oleynikov

Medical Sciences, Western University, London, ON, Canada

School of Nursing, McMaster University, Hamilton, ON, Canada

Sheila Boamah

You can also search for this author in PubMed   Google Scholar

Contributions

AG is responsible for conception and design of the review. AG, LH, CO & SB contributed to the acquisition and analysis of the data. AG, LH & SB interpreted the data. LH drafted the manuscript and AG was a major contributor to the final version of the manuscript. AG, LH, CO & SB read, provided feedback and approved the final manuscript

Corresponding author

Correspondence to Anna Garnett .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval was not required because this manuscript is a review of published literature.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Critical appraisals of included articles.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Garnett, A., Hui, L., Oleynikov, C. et al. Compassion fatigue in healthcare providers: a scoping review. BMC Health Serv Res 23 , 1336 (2023). https://doi.org/10.1186/s12913-023-10356-3

Download citation

Received : 07 August 2023

Accepted : 20 November 2023

Published : 01 December 2023

DOI : https://doi.org/10.1186/s12913-023-10356-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Compassion fatigue
• Healthcare provider
• Psychological health

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]