public administration systematic literature review

• Search Menu
• Advance Articles
• Editor's Choice
• Author Guidelines
• Submission Site
• Open Access
• About Journal of Public Administration Research and Theory
• About the Public Management Research Association
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, key claims in administrative burden research, characteristics of studies on administrative burden, qualitative analysis of key causal relationships, setting an agenda for future research, supplementary material, acknowledgment, data availability.

• < Previous

Administrative Burden in Citizen–State Interactions: A Systematic Literature Review

• Article contents
• Figures & tables
• Supplementary Data

Aske Halling, Martin Baekgaard, Administrative Burden in Citizen–State Interactions: A Systematic Literature Review, Journal of Public Administration Research and Theory , Volume 34, Issue 2, April 2024, Pages 180–195, https://doi.org/10.1093/jopart/muad023

• Permissions Icon Permissions

Based on a systematic review of 119 articles and working papers, we provide an overview of how administrative burdens in citizen–state interactions have been studied since the inception of the research agenda in 2012. We develop a new and comprehensive model of how key concepts in the framework are related, assess the evidence of the causal relationships proposed by the model, and discuss where more evidence is needed. Empirical research supports conventional claims that burdens are consequential, distributive, and constructed. However, the literature has moved further by (1) demonstrating that factors such as frontline service delivery and government communication influence experiences of burdens; (2) highlighting how factors beyond ideology influence constructions of burdens; (3) introducing the burden tolerance concept; (4) illustrating that experiences of burden influence policymakers’ and members of the publics’ burden tolerance. Based on the review, we propose an agenda for future administrative burden research. We call for studies linking experiences of burden to outcomes such as democratic behavior and take-up, and for studies connecting policymakers’ burden tolerance to actual state actions. Moreover, we argue that future studies should use qualitative methods to further explore the nature of burdens from the perspective of citizens, rely on experimental methods to establish causal links between state actions and experiences of burden, and compare burdens across contexts. Further, empirical studies should examine the tradeoffs between legitimacy and experiences of burden, and how actors outside the citizen–state interaction may influence experiences of administrative burden.

Administrative burden is defined as an individual’s experiences of policy implementation as onerous ( Burden et al. 2012 ). The concept thus emphasizes the experiences of individuals and how state actions, in the form of policies and how they are implemented in practice, influence said experiences ( Baekgaard and Tankink 2022 ). In principle, the definition applies to any individual subject to policy implementation ( Madsen, Mikkelsen, and Moynihan 2022 , 7–8), but the concept has particularly been used in the context of citizen–state interactions ( Jakobsen et al. 2016 ).

Building on research traditions on, among others, take-up of policies and benefits ( Bhargava and Manoli 2015 ; Currie 2006 ), policy feedback ( Moynihan and Soss 2014 ; Soss 1999 ), street-level bureaucracy ( Brodkin and Majmundar 2010 ; Lipsky 1980 ), and red tape ( Bozeman and Youtie 2020 ) that all draw attention to onerous experiences with the state, administrative burden has been showcased as an important concept to create an overarching framework to understand such experiences.

However, we lack a comprehensive overview of how the field has studied administrative burden since the introduction of the concept in the seminal articles by Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) , and how various research questions relate to one another. Even though the standard definition of administrative burden points to individual experiences, scholars in practice refer to different phenomena when studying administrative burden. Some focus on actions made by the state (i.e., “objective” burdens), some focus on individuals’ subjective perceptions, and some focus on individual outcomes, such as take-up of benefits or health ( Baekgaard and Tankink 2022 ). Moreover, research foci differ. Some studies focus on understanding individual experiences and outcomes and how negative experiences and outcomes can be reduced, while others focus on why policies and practices associated with burdensome experiences are enacted by policymakers or how they are implemented at the frontline.

To take stock of the current state of administrative burden research and to better connect empirical knowledge and research questions in current research, we conduct a systematic review of 119 published articles and working papers focusing on administrative burdens in citizen–state interactions. We limit our sample to papers specifically claiming to draw on this framework, that is, studies published between the inception of the concept and framework in Burden et al. (2012) and Moynihan, Herd, and Harvey (2015) and the beginning of 2023. To ensure reproducibility and transparency, we follow the widely used PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines ( Page et al. 2021 ).

There have been a number of theoretical and conceptual articles and literature reviews about administrative burdens ( Baekgaard and Tankink 2022 ; Campbell, Pandey, and Arnesen 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ; Peeters 2020 ). However, none of these articles have systematically covered all studies on the topic or taken up the task of connecting different streams of administrative burden research in a theoretical model. Our systematic review therefore makes two contributions to administrative burden research.

The first contribution is theoretical. Using a qualitative coding of the articles and papers included in our review, we build a theoretical model of how key concepts in the administrative burden causally relate to each other. This model is based partly on theoretical arguments in the literature, partly on empirical evidence, and seeks to connect studies of very different research questions within administrative burden research to create a coherent theoretical framework. The aim is not to make a parsimonious theoretical claim but rather to construct a model of the many antecedents, moderators, and potential consequences of administrative burden experiences identified in this literature.

The second contribution is an overview of how administrative burdens in citizen–state interactions have been studied to date. We describe the methodological and contextual characteristics of the studies included in the review and connect key concepts in the administrative burden framework to identify areas of inquiry where substantial progress has been made and to point to areas where future studies could best be directed.

The next section briefly discusses the concept of administrative burden and key causal claims in the administrative burden literature. In the methods section, we describe our literature search, criteria for including studies in the review, and how studies were coded and analyzed. This is followed by our qualitative analysis of the literature. We start out by presenting a model based on the review and then classify the evidence for seven causal claims in the model. The article concludes with a discussion of limitations and proposals for a future research agenda.

Administrative burden refers to the learning, compliance, and psychological costs experienced by citizens when interacting with the state ( Herd and Moynihan 2018 ). Learning costs are the costs of learning about rights, rules, and demands associated with interacting with the state ( Barnes and Riel 2022 ). For instance, an individual eligible for the TANF program in the United States has to be aware that the program exists and how to apply for the benefits. Compliance costs are the costs of complying with specific rules in interactions with the state. In the TANF example, the applicant has to fill out an application form and demonstrate eligibility. For the unemployed, compliance costs may manifest as the costs of having to show up for meetings at public offices to demonstrate an active search for work and of updating CVs on a regular basis ( Baekgaard et al. 2021 ; Madsen nda ). Finally, psychological costs have to do with the mental discomfort of interacting with the government ( Hattke, Hensel, and Kalucza 2020 ). For instance, interactions associated with uncertainty may lead to experiences of stress, loss of autonomy, or even stigma ( Cecchini nd ).

By emphasizing the subjective costs experienced by citizens and explicitly referring to individual experiences in the definition of the concept ( Burden et al. 2012 ), the administrative burden literature makes a key distinction between what the state does (sometimes called barriers, frictions, state actions, or state constructions of burdens) and what the individual experiences. However, this does not mean that the literature is only interested in the impact of experiences. Rather, the literature makes three key claims about burdens ( Herd and Moynihan 2018 ; Moynihan, Herd, and Harvey 2015 ) regarding what the state does, how citizens experience the actions of the state, individual differences in said experiences, and the consequences of burdensome experiences. Figure 1 summarizes the claims in a simplified model (see Baekgaard and Tankink 2022 ; Christensen et al. 2020 ; Herd and Moynihan 2018 for similar models).

Key Claims in the Administrative Burden Literature.

First, burdens are consequential. The impact of burdens is likely to extend beyond people’s experiences and influence outcomes such as civic and electoral participation, health, and take-up of benefits. This claim is rooted very much in literatures on policy feedback, benefit take-up, and applied economics. These research traditions empirically demonstrate that aspects of what the state does may have important impacts for the mobilization of citizens (e.g., Bruch, Ferree, and Soss 2010 ; Soss 1999 ), the extent to which target groups take up services and benefits for which they are eligible (e.g., Currie 2006 ), and the long-run health of citizens enrolled in welfare programs (e.g., Hoynes, Schanzenbach, and Almond 2016 ). However, they do less to explore people’s subjective experiences of burden. In this respect, the administrative burden framework contributes to previous streams of research by creating a language for the mechanisms linking state actions to outcomes.

Second, burdens have distributive consequences and are likely to fall harder on those with fewer resources in the form of human and administrative capital ( Christensen et al. 2020 ; Masood and Nisar 2021 ), administrative literacy ( Döring 2021 ), and bureaucratic self-efficacy ( Bisgaard 2023 ). Third, while some burdensome state actions are likely unintended ( Peeters and Widlak 2023 ), or the result of unconscious biases ( Olsen, Kyhse-Andersen, and Moynihan 2020 ), other state actions are constructed and the result of deliberate political and administrative decisions, that is, politicians or bureaucrats prefer to introduce burdens to, for instance, limit fraud in public programs ( Moynihan, Herd, and Ribgy 2016 ).

Thus, the literature relies on a broad understanding of the relevant domain of inquiry. This domain is not limited to the study of experiences and outcomes among citizens and target groups. As per the third causal claim above, it also encompasses decisions and rationales for decisions made by elected politicians, administrators, and frontline personnel. In our review, we therefore rely on a broad understanding of the domain of administrative burden in citizen–state interactions where studies are relevant whenever the subject matter has to do with a state arrangement introducing burden for citizens. The studies may focus either on decisions made by politicians or bureaucrats or on the consequences of such decisions for citizens. In our analysis of the literature, we seek to develop the model presented in figure 1 further by reviewing the empirical findings in existing studies.

We adhere to the PRISMA guidelines when conducting our systematic literature review ( Page et al. 2021 ). These guidelines were developed to ensure that literature reviews are comprehensive, transparent, and well documented to minimize reporting biases and ensure reproducibility. The PRISMA checklist is available in the appendix . Below we describe the eligibility criteria for inclusion in the review as well as our search and coding strategy.

Eligibility Criteria

Our focus is on administrative burdens in citizen–state interactions. The main inclusion criterion is that studies use the conceptual framework formulated by Burden et al. (2012) , Moynihan, Herd, and Harvey (2015) , and Herd and Moynihan (2018) , that is, refer explicitly to administrative burden and/or learning, compliance, and psychological costs. Other streams of literature in economics, sociology, political science, and public administration also deal with frictions in interactions between citizens and government. This includes, but is not limited to, literatures on red tape, sludge, ordeals, take-up of government benefits, street-level bureaucracy, and policy feedback ( Baekgaard and Tankink 2022 ; Madsen, Mikkelsen, and Moynihan 2022 ). However, as administrative burden has developed into a sizeable subfield of its own, which in several aspects differs from related research in other disciplines ( Madsen, Mikkelsen, and Moynihan 2022 ), it is important to take stock of the current state of this particular field and explore what the literature has taught us so far.

The second inclusion criterion is that studies focus on administrative burdens in citizen–state interactions. This means that we exclude studies that use the administrative burden framework but focus either on companies ( Petersen, Hansen, and Houlberg 2022 ), third-party organizations ( Carey et al. 2020 ), or on the costs experienced by public employees in their interactions with the organization where they are employed ( Bozeman and Youtie 2020 ; Linos and Riesch 2020 ; Sievert, Vogel, and Feeney 2020 ). We make this decision because our goal is to understand how, why, and when citizens experience burdens in their interactions with the state. In comparison, studies on public employees burdened by work routines focus on internal organizational affairs rather than a bureaucratic relationship between the state and individual outside the formal organizational hierarchy. Also, burdens among public employees have been studied extensively in the red tape literature ( George et al. 2020 ). Nevertheless, the review still includes studies where elected politicians and frontline personnel were asked about the imposition of burdens on individuals outside the organization. Thus, the review applies a broad understanding of citizens as individuals and organizations outside the formal organizational hierarchy of the state in a given case.

The remaining inclusion criteria are more straightforward. We are interested in all English-language peer-reviewed publications and working papers from 2012 until our data collection closed in February 2023. 1 We set the start to 2012, because this is when Burden et al. (2012) wrote their seminal article that introduced and defined the term “administrative burdens.” Table 1 gives an overview of the eligibility criteria.

Overview of Eligibility Criteria

Literature Search

To identify peer-reviewed journal articles, we searched all journals in the Social Sciences Citation Index using Web of Science. We searched titles, abstracts, and keywords for “administrative burden,” “psychological cost,” “compliance cost,” “learning cost,” and derivatives of these terms. We limited our search to English-language articles. We also searched 12 leading public administration journals (see list of journals in appendix table A1 ) using the same terms. We then screened titles and abstracts and then full papers to identify all papers that passed our eligibility criteria. Finally, we screened the literature list of all eligible journal articles for missing records. In total, we identified 100 peer-reviewed journal articles for the systematic literature review.

To obtain a comprehensive pool of working papers, we created a list of all authors who contributed at least two articles to the literature review (see appendix table A2 ). We then contacted all authors on the list and asked them to provide any unpublished, full-length papers on administrative burdens that they had (co-)authored. We also encouraged them to let us know if they knew of other working papers on the topic. Almost all authors replied within a few days and most sent one or more working papers. Moreover, we made a call for working papers through our Twitter accounts and a similar call through a listserv for scholars interested in administrative burden research managed by Professor Donald Moynihan. Based on these steps, we collected 19 eligible working papers. 2 In total, 119 papers are included in the review (see the full list of papers in appendix table A7 ). Figure 2 summarizes the selection process.

Selection Process.

Coding Strategy

We relied on two strategies for coding the articles. First, we systematically coded several facts about the articles (year of publication, whether empirical material was collected, methods used, country covered by empirical analysis, policy area, and type of subjects) using a closed coding strategy (see appendix table A3 for a full description of coding criteria). We present this information in the first part of the results section to give an overview of the field and the types of studies conducted.

Second, we used an open coding where we focused on core concepts covered in the articles and types of causal relations covered in the papers. This is a demanding task that requires that coders have in-depth knowledge of the literature. We therefore handled all coding ourselves and met several times during the coding process to ensure consistency in the categorization of relationships and concepts. We use the qualitative coding to summarize current knowledge about the different relationships shown in figure 1 and to extend the causal model based on the findings and arguments in extant research.

Citation Analysis

This first part of the analysis covers key characteristics of the articles on administrative burdens in citizen–state interactions. Related to the discussion of eligibility criteria, we initially explore whether studies frequently cited by our 119 eligible studies are missing in the review. Table 2 shows that among the top 10 most cited papers and publications in the review, three publications do not rely on the administrative burden framework and therefore do not meet the eligibility criteria. Two of these publications ( Brodkin and Majmundar 2010 ; Lipsky 1980 ) concern street-level bureaucracy, and the third ( Bhargava and Manoli 2015 ) focuses on take-up of benefits. Thus, while there certainly are some widely cited works outside the narrow domain of administrative burden research, the field is generally dominated by internal references, suggesting that administrative burden research indeed constitutes a distinct field of its own.

Top 10 Most Cited Publications by the 119 Papers Included in the Systematic Review

A related question is how well studies with different foci, research questions, and methodologies speak to one another. We conducted a bibliographical network analysis ( Perianes-Rodriguez, Waltman, and van Eck 2016 ) in which we explored citation patterns between articles. As shown in appendix table A4 , assortativity scores are generally low, suggesting that articles tend to cite each other to an almost equal extent despite different methodologies and research questions ( Newman 2003 ). Overall, the analysis suggests that the field is coherent in the sense that even the most different parts of the field tend to rely on each other’s work.

Methodological Characteristics

Of the 119 collected articles, 75% are empirical papers using qualitative or quantitative analysis of data, while 25% are theoretical papers, literature reviews, or case studies. Articles are published in 35 different journals. Most are published in public administration journals, but some are published in either health, economics, or political science journals. The most frequent appearances are in Public Administration Review with 17, Journal of Public Administration Research and Theory with 16, and Journal of Behavioral Public Administration with 11 articles (see appendix table A5 for full details). Figure 3 shows a timeline of all published papers on the topic. Only nine were published between 2012 and 2017, but the publication trend changed significantly in 2018. From 2018 to 2021, the number of yearly published papers almost doubled each year from 5 in 2018 to 36 in 2021. While 2022 saw a decline in publications to 20, the overall trend still indicates that the study of administrative burden has established itself as a sizeable subfield within public administration research.

Publication Timeline.

Note: n = 100. The figure shows the year studies were made available online and does not include working papers.

Figure 4 graphs methodological characteristics of the studies. Panel A shows that more than half the empirical studies use quantitative methods. However, a substantial number of articles employ qualitative methods or case studies, meaning the field is characterized by some methodological diversity. This is also evident from panel D, where we divide the quantitative and qualitative categories into more specific subcategories. We see that studies on administrative burdens use a great variety of methods, and that studies utilize both observational and experimental data to a high extent. Studies are also relatively diverse when it comes to the origin of data, as our review includes studies from all six inhabited continents. However, studies from Western countries dominate the literature, as 82% of all studies were conducted in either the United States, Europe, or Australia (see panel B). We also coded whether papers used data from more than one country. Only three papers used data do so, and neither of them used a comparative approach where they compared burdens across contexts. Panel D shows that almost half of the studies focus on target group members. This aligns well with the fact that one purpose of the administrative burden framework is to draw attention to individuals’ experiences of policy implementation ( Moynihan, Herd, and Harvey 2015 ). Finally, panel E shows that around 50% of all studies focused on means-tested welfare benefits. This may reflect that means-tested programs are often where citizens encounter the most requirements and therefore are likely to experience various burdens when interacting with the state.

Methodological Characteristics of Empirical Studies.

Note: Figures A-E display various charatersitics of empirical studies. Articles that fit into more than one category are coded into all relevant categories. Purely theoretical articles are not included in any of the figures.

This section presents the results of our qualitative analysis of the literature. Figure 5 provides an overview of our main findings. This model extends the theoretical model in figure 1 in four important respects. First, it proposes a more nuanced understanding of what state actions are. In line with Baekgaard and Tankink (2022 , 17), we understand state actions broadly to cover what the state does “including laws, rules, requirements, and how such are implemented by public officials and street-level bureaucrats.” This leads us to distinguish between formal (arrow 1) and informal policy designs (arrow 2). While formal policy design refers to the laws and rules enacted by politicians, that is, the rules that people will have to abide to get access to services and benefits, informal policy design concerns how these rules are implemented at the frontline and communicated more broadly. This allows us to discuss how different aspects of policies lead to experiences of administrative burdens. Second, the model extends the number of factors explaining state actions beyond political ideology by introducing the concepts of burden support and burden tolerance, that is, “the willingness of policymakers and people more generally to passively allow or actively impose state actions that result in others experiencing administrative burdens” ( Baekgaard, Moynihan, and Thomsen 2021 , 184). As shown, support and tolerance for burdens may sometimes be influenced by the content of state actions when people become aware of actual rules and implementation (arrow 6b).

Extended Model of Causal Claims.

Third, the model proposes that other factors than political ideology and beliefs may influence burden tolerance and state actions. In particular, the model highlights the importance of target group deservingness, personal experience, and bureaucratic processes (arrow 7). Fourth, the model proposes feedback effects of citizens’ experiences of burden on how burdens are constructed by the state and how tolerant policymakers and others are of burdens to begin with (arrows 5a and 5b).

Table 3 lists the number of studies that cover each relationship. Below, we discuss each of the seven arrows in figure 5 . Our aim is not to mention all studies discussing each specific arrow but rather to summarize current knowledge about each relationship. Our discussion therefore only covers selected articles that provide knowledge on the relationship under discussion. Appendix table A6 is an extended version of table 3 and shows the articles that provide knowledge on each relationship.

Number of Papers Studying Each Causal Relationship

Arrow 1: Formal Policy Design → Experiences of Burden

With few exceptions, studies find that state barriers are associated with experiences of learning and compliance costs. Learning costs, for instance, arise when being subject to requirements ( Cook 2021 ), misinformation ( Chudnovsky and Peeters 2021a ), and having to deal with vouchers ( Barnes 2021 ), while compliance costs arise because of transportation time to vaccinator camps ( Ali and Altaf 2021 ) and completing forms ( Yates et al. 2022 ). Some studies find that learning and compliance costs arise as a consequence of (eligibility) requirements in means-tested welfare programs ( Holler and Tarshish 2022 ) and insurance programs ( Yates et al. 2022 ). Other studies find that learning and compliance costs also arise in settings such as the restoration of voting rights ( Selin 2019 ), digital government services ( Madsen, Lindgren, and Melin 2022 ), and accessing vaccinations ( Ali and Altaf 2021 ).

Studies are conducted in diverse contexts such as Pakistan, Denmark, the United States, and Argentina, suggesting there is some universality to the claim that interacting with the state is associated with experiences of learning and compliance costs. However, one paper finds that having a scheduled compulsory meeting with frontline workers causes no changes in compliance costs and is associated with experiences of less learning costs ( Baekgaard and Madsen 2023 ). Another study finds that digital self-service solutions have the potential to both increase and reduce learning and compliance costs ( Madsen, Lindgren, and Melin 2022 ).

This suggests that more research is needed on how different types of state actions reduce and impose experiences of learning and compliance costs. Such studies could build on more qualitative approaches to obtain a better understanding of the mechanisms linking state actions to experiences. Also, when it comes to understanding the costs of dealing with different state actions, qualitative methods have major advantages over other methods. With a few exceptions ( Ali and Altaf 2021 ; Baekgaard and Madsen 2023 ), most papers indeed use qualitative methods to study the relationship between barriers and learning and compliance costs, while no papers use experimental methods. This is not surprising, as it is often hard to manipulate barriers or state actions. However, in addition to more qualitative research, the literature would benefit from studies that are able to causally link state actions to experiences of learning and compliance costs. As mentioned in the next section, a few studies document how state actions causally influence experiences of psychological costs, showing that it is possible to causally study the link between state actions and experiences of administrative burdens.

There are 50% more studies on the relationship between formal policy designs and psychological costs than on the comparable relationship with learning and compliance costs discussed above, illustrating that this relationship has received high scholarly attention. The general finding from the 16 studies discussing this topic is that state actions are associated with various forms of psychological costs. Examples of psychological costs arising from state actions are autonomy loss and stress ( Baekgaard et al. 2021 ), frustration ( Cook 2021 ), stigma ( Selin 2019 ; Thomsen, Baekgaard, and Jensen 2020 ), externalization of locus of control ( Madsen and Mikkelsen 2022 ), uncertainty ( Cecchini nd ) and confusion, anger, and frustration ( Hattke, Hensel, and Kalucza 2020 ).

Studies fall in two methodological categories: qualitative studies and experiments. Qualitative studies provide in-depth knowledge about how state actions may lead to psychological costs. One example is Yates et al.’s (2022) study of burdens in Australia’s National Disability Insurance Scheme. One interviewee mentions that it was “wearing” and “soul destroying” “to be constantly questioned about, are you disabled enough” (p. 5), showing how eligibility requirements can create psychological costs.

Experimental studies establish causal links between barriers and costs. Baekgaard et al. (2021) use survey- and field-experimental evidence to show that reductions in state compliance demands reduce stress and increase the sense of autonomy among target group members. Hattke, Hensel, and Kalucza (2020) and Hattke et al. (nd) rely on laboratory experiments to show how redundant documentation requirements and simple administrative processes can cause confusion, frustration, and anger.

In general, the link between state actions and psychological costs is relatively well covered in the literature. However, studies so far have generally examined only one or a few state actions. There is a lack of studies that compare effects of different actions on psychological costs. Such studies could provide valuable knowledge on which state actions translate into psychological costs.

Arrow 2: Informal Policy Design → Experiences of Burden

Informal policy design has to do with the actions by the state that do not directly refer to the formal rules and requirements as decided by policymakers but rather how these are processed and communicated to citizens. Two aspects of informal policy design are particularly prevalent in research on administrative burden: frontline service delivery and government communication.

Frontline Service Delivery

It is no surprise that the delivery of services at the frontline of public organizations matters for experiences of burden. Lipsky (1980) alluded to this, and subsequent work has explored this question without explicitly using the concept of administrative burden (e.g., Brodkin and Majmundar 2010 ; Soss, Fording, and Schram 2011 ). Studies applying the administrative burden framework show that workload matters for experiences of administrative burden. For instance, Bell and Meyer (nd) use administrative data from college financial aid programs to show that decreases in workload lead to an increase in program access for low-income students and that the increase is highest among students who have been subject to discrimination based on their race. Ali and Altaf (2021) show that citizens experience more burdens in areas with lower administrative capacity, while others find that stress and burnout ( Mikkelsen, Madsen, and Baekgaard 2023 ) and red tape ( Madsen ndb ) among frontline workers are associated with experiences of burden among their clients.

The behavior of frontline workers also matters for citizens’ experiences. Bell and Smith (2022) show that frontline workers who adopt a support role rather than a role as “compliance officer” are more likely to use their discretionary power to help students overcome administrative burdens. In a similar vein, Halling’s (nd) results suggest that frontline workers help citizens overcome burdens by circumventing rules. Finally, Barnes and Henly’s (2018) qualitative analysis shows that clients tend to blame their experiences of administrative burden on frontline employees.

Government Communication

Another part of informal policy design that has received considerable attention is how communication from the state affects individuals’ experiences of administrative burden. All these papers rely on field experiments with randomized exposure to different forms of government communication. Linos et al. (2022) show that disadvantaged groups prefer postcards over a telephone hotline to seek information about free dental care. They use focus groups to show that this is likely explained by lower psychological costs associated with postcards as participants fear uncomfortable interactions with bureaucrats. Moynihan et al. (2022) show how the framing of state categories matters for selecting into the right categories and that a more intuitive presentation of information increased the number of claimants providing adequate documentation. Simplified communication ( Linos, Reddy, and Rothstein 2022 ), destigmatizing language ( Lasky-Fink and Linos 2023 ), early communication ( Linos, Quan, and Kirkman 2020 ), postcards ( Hock et al. 2021 ), letters ( Bhanot 2021 ), and text messages ( Lopoo, Heflin, and Boskovski 2020 ) can also improve take-up.

Altogether, these field experiments show that different forms of nudges can be effective in increasing take-up of benefits among eligible individuals. Apart from the two first-mentioned studies, the studies do not measure experiences of burden directly. Instead, they measure different outcomes while theorizing that the link between communication and outcomes has to do with experiences of burden. Hence, there is a need for studies that show that reduction of administrative burdens is the process through which these nudges work.

Arrow 3: Distributive Effects

The argument that administrative burdens are distributive and can foster inequality is at the core of the administrative burden framework ( Christensen et al. 2020 ; Herd and Moynihan 2018 ). Thirty-one papers contribute knowledge on the distributional consequences of state actions. Differences in resources, attitudes, and expectations between citizens constitute one main type of distributive effects identified in the literature ( Christensen et al. 2020 ; Heinrich 2018 ; Nisar 2018 ). The other type, which has received less attention, focuses on how characteristics of the state may contribute to different experiences of burden among different parts of the population ( Griffiths 2021 ; Peeters, Renteria, and Cejudo nd ). We discuss both types next.

Citizen Factors

Studies show that possessing administrative literacy ( Döring 2021 ; Döring and Madsen 2022 ), self-efficacy ( Thomsen, Baekgaard, and Jensen 2020 ), habitus and different forms of capital ( Carey, Malbon, and Blackwell 2021 ; Masood and Nisar 2021 ) all make state barriers easier to handle, resulting in fewer experiences of burdens. All these contributions are important in documenting that possessing the necessary capital and skills is key when dealing with onerous state demands.

However, there is a considerable overlap between the different concepts. Apart from self-efficacy, all focus on a type of capital (or literacy) that makes state encounters easier to handle. Some are specific to encounters with the state (administrative literacy and capital), while others are more general forms of capital (human capital and Bourdieu’s capital concepts). Discussing differences and similarities between the concepts is beyond the scope of this article, but we note that using fewer concepts would strengthen the comparative potential across studies.

Other studies focus on how experiences of burdens are distributed across demographic and non-demographic characteristics. The general finding is that individuals from marginalized or low-resource groups tend to struggle more with state barriers. So far, studies have shown that individuals with low income or who are experiencing scarce financial resources ( Chudnovsky and Peeters 2021b ; Heinrich et al. 2022 ; Larsson 2021 ; Madsen, Baekgaard and Kvist 2022 ), ethnic minorities ( Heinrich 2018 ; Olsen, Kyhse-Andersen, and Moynihan 2020 ), women ( Kyle and Frakt 2021 ; Yates et al. 2022 ), individuals with low or no education ( Chudnovsky and Peeters 2021b ; Collie et al. 2021 ; Kyle and Frakt 2021 ), and those suffering from sickness and disabilities ( Bell et al. 2022 ; Collie et al. 2021 ; Kyle and Frakt 2021 ) experience more administrative burdens as a result of state actions.

Relatedly, a few studies discuss how citizens’ attitudes and expectations might influence how citizens engage with the state and hence lead to different impacts of state actions on experiences of burden. These attitudes and expectations may themselves stem from a variety of sources including prior interactions with the state ( Chudnovsky and Peters 2021b , 531), thus suggesting a potential feedback effect from outcomes on attitudes and expectations (see also Moynihan and Soss 2014 ). 3

Finally, a last stream of studies considers how individuals’ access to relevant third parties, actors outside the citizen–state interaction that provide help to citizens or otherwise influence interactions ( Moynihan, Herd, and Harvey 2015 ), may affect their experiences of administrative burden. A few papers explore the role of such actors. Barnes (2021, nd ) shows that retailers play a crucial role in shaping compliance costs in voucher programs such as WIC. Because citizens must redeem their vouchers in retail stores, retailers play a huge role in shaping how easy redemption is. Concrete examples are the degree to which eligible food is marked and displayed and whether store personnel are trained in handling vouchers. NGOs may also contribute to reduced learning and compliance costs by helping citizens overcome burdens ( Nisar 2018 ; Nisar and Masood nd ). Finally, (ex-)family members may influence experiences of administrative burden ( Nisar 2018 ). Cook (2021) illustrates how ex-partners may directly impose burdens on mothers in the child support benefit system in Australia. As an example, some fathers limit their child support liabilities or claim that they have already provided payments to mothers. Each time fathers make such changes or claims, mothers are required to respond, which can be associated with substantial compliance costs.

State Characteristics

Another possible source of distributive effects is the state itself. A key insight from this stream of research is that variations in administrative capacities to reach out to vulnerable populations may contribute to inequality in the experience of burdens. Some studies investigate how individuals may experience different burdens in states with different characteristics. The most prominent characteristic examined so far is the extent to which the state is automated and digitalized. Peeters, Renteria, and Cejudo (nd) illustrate how governments with higher information capacity are better able to “absorb” burdens, which means that citizens face fewer administrative burdens. Digital government may also create unintentional errors that contribute to considerable experiences of administrative burden. Griffiths (2021) shows how automation of benefit calculation can create burdensome experiences. For example, people with irregular pay dates risk missing out on benefits for which they are eligible because automation processes do not account for irregular cases. Likewise, Widlak and Peeters (2020) show that citizens face various administrative burdens in correcting errors made by the state, while Compton et al. (2022) show that blacks and Hispanics are disproportionally hit by administrative errors.

Other state characteristics that may influence experiences of administrative burdens are material and artificial artifacts present in physical and virtual government arenas ( Nisar and Masood nd ) and consistent application of rules ( Kaufmann, Ingrams, and Jacobs 2021 ). Finally, Johnson and Kroll (2021) theorize but find no supporting empirical evidence that representative government and shared identities between frontline employees and citizens may decrease experiences of burden.

Arrow 4: Experiences of Burden → Outcomes

According to Moynihan, Herd, and Harvey (2015) , administrative burdens are an important part of governance, “since they affect whether citizens succeed in accessing services (did I get what I want), whether public polices succeed (did a program reach the targeted group?), and the perceptions of government (was I treated fairly and with respect?)” (p. 43). However, despite the obvious importance of studying the link between experiences of burden and various outcomes, only Daigneault and Macé (2020) have done so among published papers. Based on interviews with target group members, they show that individuals experiencing compliance and learning costs are less likely to take up Quebec’s Supplement to the Work Premium program. Other papers study the link between state actions and outcomes but without subjective measures of people’s experiences of administrative burden. Notable examples are Heinrich (2016) and Jenkins and Nguyen (2022) , who convincingly, and with strong causal traction, show that various state actions influence take-up of welfare programs and might even impact long-term outcomes such as risky behaviors in adolescence ( Heinrich 2016 ; Heinrich and Brill 2015 ). These studies contribute important knowledge on how state actions influence take-up of welfare benefits but not on the relationship between subjective experiences of burden and outcomes.

Several working papers show that experiences of burden are associated with behaviors that can lead to reduced program take-up, such as compliance and autonomous motivation ( Madsen nda ), making errors on forms ( Hattke et al. nd ), and filing complaints ( Bell et al. 2022 ). While these papers make valuable contributions, none of them study actual outcomes but rather behaviors that are likely to influence take-up of benefits. The final working paper by Lasky-Fink and Linos (2023) offers a promising approach to dealing with some of the shortcomings of other research on this relationship. Contrary to the other working papers, the authors study actual take-up of welfare benefits and show that destigmatized language leads to substantially higher take-up rates. Moreover, contrary to studies linking state actions and take-up, the authors go one step further and use three survey experiments to make it probable that the mechanism linking state actions and take-up is psychological costs in the form of perceived stigma. In doing so, the working paper studies the whole causal chain from barriers over subjective experiences of administrative burdens to outcomes. This is a model for future studies to pursue because such studies will be able to show not only whether individuals experience burdens as a result of state actions, but also the extent to which these burdens subsequently influence service use or other relevant outcomes.

There is also a lack of studies that look beyond take-up and focus on other types of outcomes. In some instances, burdens may not discourage people from taking up public services, but they may still affect the adequacy and quality of services provided—in particular when citizens interact with the same public agency for a prolonged period of time ( Peeters and Campos 2021 ). Furthermore, inspired by the policy feedback literature, it has been suggested that experiences of burden may affect civic capacities such as political efficacy, trust in institutions, and civic engagement ( Christensen et al. 2020 ). However, no studies have so far examined these questions systematically.

Arrow 5: Feedback Effects: Experiences → Burden Tolerance and State Actions

While state actions are expected to trigger experiences of burdens in the original theoretical model, a few studies suggest a feedback effect, that is, experiences may influence burden tolerance and state actions. The argument is that knowledge about experiences may make policymakers and others understand the detrimental effects of state actions and hence induce less burden. This proposition finds mixed support in the three studies dealing with the question. In a survey-experimental study of Danish local politicians using a treatment cue about psychological costs experienced by target group members, Baekgaard, Moynihan, and Thomsen (2021) find no evidence of a feedback effect. Conversely, in a survey experiment, Halling and Petersen (nd) find that Danish frontline employees are more likely to reduce compliance demands in the implementation process and to help citizens who communicate psychological costs. Sievert and Bruder (2023) find mixed support in their study of the feedback effects of treatments increasing awareness of learning and compliance, costs among German citizens. While there is some evidence of feedback effects of compliance costs, exposing participants to information about learning costs does not affect burden tolerance. Finally, Gilad and Assouline (2022) do not study feedback effects directly, but rather a prerequisite of their existence, namely citizens voicing their experiences of burden. They find that citizens indeed voice their experiences to authorities but also that disadvantaged groups are less inclined to do so.

On balance, there is a need for much more research to establish the relevance of feedback effects. Such studies could investigate differences between groups of respondents (policymakers, frontline workers, citizens). They may also focus on the way in which information about experiences of burden is provided. Here, a distinction could be made between statistical and episodic information. Previous research has identified stronger effects of episodic data in other contexts ( Olsen 2017 ). Finally, studies could examine feedback effects from citizen outcomes.

Arrow 6: The Relationship Between Burden Tolerance and State Actions

The literature on burden tolerance presumes that tolerance among political decision-makers and the mass public influences the extent to which the state constructs burdens (e.g., Aarøe et al. 2021 ; Baekgaard, Moynihan, and Thomsen 2021 ; Keiser and Miller 2020 ; Nicholson-Crotty, Miller, and Keiser 2021 ). However, none of the studies in the review study the causal influence of burden tolerance on state actions, likely due to challenges obtaining causal estimates. Nevertheless, we indicate this relationship in figure 5 with a dashed line (arrow 6a) due to the strong theoretical expectation that burden tolerance influences the extent to which the state introduces burdens in public policies.

Alternatively, it is possible that knowledge about existing barriers influences the extent to which people are supportive of burdensome barriers (arrow 6b). Two empirical studies examine this question using survey experiments among the mass public. Keiser and Miller (2020) find that information about the presence of barriers increases support for welfare programs and their recipients, in particular among conservative voters. Nicholson-Crotty, Miller, and Keiser (2021) show that information about barriers has heterogeneous effects on program approval depending on whether the target group is perceived as deserving (information about more barriers reduces approval) or undeserving (information about barriers has no significant effect). While the two studies support the idea that information about state actions may influence burden tolerance, there is certainly room for more research about how state actions may influence burden tolerance in the mass public and among decision-makers. Such studies may for instance investigate how state actions are constructed in popular debates.

Arrow 7: Factors Shaping Burden Tolerance and State Actions

This section looks into other factors that shape burden tolerance and state actions. A total of seven studies examine factors shaping burden support, while 13 studies investigate factors shaping state actions. We deal with the questions jointly, because many of the key explanations are similar for burden tolerance and state actions. Overall, explanations can be divided into four broad categories.

First, a series of studies present evidence that burdens are constructed and that political ideological beliefs influence the extent to which barriers are introduced. For instance, the studies by Moynihan, Herd, and Harvey (2015) , Moynihan, Herd, and Ribgy (2016) , and Heinrich (2018) find that more barriers are introduced in states governed by conservatives than in states governed by liberals. Likewise, a series of cross-sectional studies find strong correlations between the ideological beliefs of politicians ( Baekgaard, Moynihan, and Thomsen 2021 ), street-level bureaucrats ( Bell et al. 2020 ), and the mass public ( Haeder, Sylvester, and Callaghan 2021 ; Halling, Herd, and Moynihan 2022 ) and their support for administrative burden policies.

Second, in accordance with the claim by Schneider and Ingram (1993) that target group construction matters to the benefits and burdens assigned to each group, target group deservingness and minority status appear to be of major importance to both burden tolerance ( Baekgaard, Moynihan, and Thomsen 2021 ; Haeder, Sylvester, and Callaghan 2021 ) and barriers ( Jilke, Van Dooren, and Rys 2018 ).

Third, a series of individual-specific explanations of burden tolerance have been investigated in the literature. Most factors have not been theorized very clearly, however, and have only been the subject in few empirical studies. Personal experience with benefits has been shown to be associated with less tolerance for burdensome state actions among Danish local politicians ( Baekgaard, Moynihan, and Thomse 2021 ) and a representative sample of US citizens ( Halling, Herd, and Moynihan 2022 ), while big five personality traits in the form of conscientiousness and openness to experiences have been shown to correlate with burden tolerance in the study of Aarøe et al. (2021) .

Fourth, studies of factors explaining variation in barriers find bureaucratic processes are likely to shape the barriers that citizens meet when interacting with the state. These studies are primarily based on discussions of specific exemplary cases. Peeters (2020) points out that barriers are likely to be unintentional in many cases. They can, for instance, be a result of very complex cases that make it impossible to ease application processes for citizens by means of automation ( Larsson 2021 ), or they can be unintended results of large-scale digitalization and automated decision-making processes where citizens who do not fit into predefined boxes face barriers in the implementation process ( Peeters and Widlak 2018 , 2023 ). Other studies show that bureaucratic low-trust culture and inertia may increase barriers that citizens face when interacting with government ( Bashir and Nisar 2020 ; Peeters et al. 2018 ).

Before we move on to the discussion of next steps to be taken, we note three limitations of our study. The first is publication bias. While we approached the field to include unpublished research, it is possible that some unpublished null findings have not been included or that published null findings did not show up in our literature search because publications with null findings on administrative burden hypotheses have been framed into other literatures. While we consider this a lesser concern given our extensive strategy for collecting studies, publication bias may have made evidence appear stronger than it is. The second limitation has to do with the qualitative coding of studies. While we adhere to stringent coding criteria and have conducted multiple rounds of cross-validating the coding, categorizing studies based on the kind of relationships they study is—at least for some studies—a matter of nuance and assessment. Third, the quality of the included studies is likely to vary, meaning that our review may not give an accurate picture of the strength of evidence for the many propositions studied in administrative burden research. While we have confidence in the general pattern of how different relationships have been covered, others may disagree with our coding of some studies and with the strength of evidence presented in these studies.

Limitations aside, our review points out where evidence is missing and suggests steps to be taken in future research. Next, we discuss which parts of our theoretical model warrant more empirical evidence before finishing with a discussion of new questions for future research to pursue.

More Evidence Needed

Our review points to several issues that should get more attention in future research. First, our understanding of people’s experiences is very much based on the deductive categorization of experiences as learning, compliance, and psychological costs developed in Moynihan, Herd, and Harvey (2015) . While this has laid the foundation for important research, future research could do more to supplement it with bottom-up qualitative research of what burdens are from the perspectives of those interacting with the state. Such research could also aid our understanding of what constitutes more important types of burdensome experiences and under what circumstances they arise. A good example of this kind of research is the work of Barnes (2021, nd ).

Second, it is a core claim of the administrative burden framework that what the state does is consequential for citizens’ experiences. Providing solid causal evidence about this relationship is therefore a key point for future research. Future studies could for instance rely on laboratory experiments inspired by the studies by Hattke, Hensel, and Kalucza 2020 and Hattke et al. (nd) . Another way forward may be to embed surveys and in-depth interviews as part of randomized field experiments to explore how changes in state action influence experiences and in turn outcomes. Here, the study by Lasky-Fink and Linos (2023) may also serve as an example to follow, as the authors combined their field experiment with survey experimental evidence to explore whether the impact of destigmatized language on take-up indeed was mediated by reduced perceived stigma as hypothesized by the authors.

Third, most studies examining this link are conducted among recipients of various social welfare benefits. However, experiences of burden are likely to arise in other types of interactions with the state as is evident from studies of, among others, digital government services ( Madsen, Lindgren, and Melin 2022 ) and voting rights ( Herd and Moynihan 2018 , 43–70; Selin 2019 ). To better understand the scope and importance of administrative burden, there is a need for studies that move beyond social welfare to investigate experiences of burdens in areas such as law enforcement, taxation, and regulation.

Fourth, research on how experiences of administrative burden affect outcomes such as welfare take-up, trust in government, health, and voting behavior is scarce. Most of the articles that study outcomes (primarily take-up) examine how they relate to state actions and not to experiences of burden. To get a more comprehensive picture of how burdensome encounters influence citizens’ lives, we encourage future studies to examine the link between experiences of burden and outcomes.

Fifth, the advancement of the burden tolerance concept allows researchers to examine the extent to which individuals support barriers. An important assumption is that the burden tolerance of policymakers and bureaucrats shapes the actual design of state actions, but it has never been empirically examined. Doing so would help ascertain whether burden tolerance is consequential for the actual design of polices.

Sixth, the administrative burden literature is diverse in terms of methods, policy areas, and subjects. Most studies are conducted in Western countries, but there are studies of burdens from other contexts such as Pakistan and Latin America. However, there is a general lack of comparative studies of burdens across countries and across policies, which would be valuable in terms of providing knowledge on the extent to which context matters for experiences of burden. Likewise, comparative studies of barriers or across policy areas could elucidate which types of state actions are most likely to produce experiences of burdens.

New Questions to Pursue

While we have presented a quite extensive model based on current administrative burden studies, there are still important questions that have received little to no attention in the literature. An important part of the framework formulated by Herd and Moynihan (2018) is that burdens are not inherently bad, and that they often serve legitimate purposes of protecting program integrity and avoiding fraud. While the issue of burden legitimacy has received some theoretical attention ( Doughty and Baehler 2020 ), empirical scholarship has yet to engage with it. One important question is how policymakers and citizens form preferences regarding program integrity vis à vis target group members’ onerous experiences. Studies on burden tolerance touch upon this question, but do not tackle it directly. Another question is how policymakers legitimize the existence of administrative burdens. Do they emphasize fraud protection, budget concerns, targeting the most deserving individuals, or something else? A third question that should get more attention is how actors outside the citizen–state interaction shape experiences of administrative burdens. A few studies show that various third parties such as NGOs and family members can influence experiences of burden, but the roles of these actors still warrant more attention. Further, civil society and the media may influence citizens’ experiences. For example, target group members are often negatively portrayed in the media ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), which could increase their experiences of burden.

The administrative burden literature, while surprisingly clearly demarcated from other fields of research, has developed into a thematically and methodologically diverse research field within few years. Overall, our systematic review demonstrates that empirical research in the field generally supports the original three-fold claim made by Moynihan, Herd, and Harvey (2015) that burdens are consequential, constructed, and fall harder on groups with few resources. Yet, the review also demonstrates that the literature has moved past these claims in important ways. Based on our reading and coding of 119 articles and working papers, we build a comprehensive model of causal claims in the literature. The model illustrates different relationships that have been explored in the still nascent literature on administrative burdens, and it highlights several new theoretical insights gained since the founding work of Moynihan, Herd, and Harvey (2015) . First, experiences of administrative burdens are sometimes unrelated to how burdens are constructed by the state and instead rely on other factors such as frontline service delivery, government communication, unintended actions, and third parties. Second, the model highlights that factors beyond political ideology may affect the construction of state actions by introducing the concept of burden tolerance. Third, the model shows that factors such as personal experience with programs, personality traits, and the structure of bureaucratic processes affect individuals’ burden tolerance. Finally, the model illustrates a potential feedback effect of citizens’ experiences of administrative burden on policymakers’ burden tolerance.

Our systematic coverage of the administrative burden literature offers promising avenues for new research. First, we call for studies that causally link state actions and experiences of administrative burden, for studies that link experiences of burden to outcomes such as democratic behavior and take-up, and for studies that connect policymakers’ burden tolerance to actual state actions. Methodologically, we call for in-depth qualitative studies of how burdens are experienced by people taking part in citizen–state interactions and comparative studies. Last, we argue that important questions remain unexplored. One topic that future research should address is how policymakers, bureaucrats, and members of the public balance the legitimacy of public policies against target group members’ experiences of administrative burden. Is it acceptable to enhance experiences of administrative burdens to avoid fraud or to target the right populations? Another topic that warrants more attention is how actors outside the citizen–state interaction shape experiences of administrative burden. For example, we know that welfare recipients are often negatively constructed in the media and society ( Baekgaard, Herd, and Moynihan 2022 ; Schneider and Ingram 1993 ), yet we have limited knowledge about whether this leads to them experiencing administrative burdens to a larger extent when interacting with the state.

Supplementary data is available at the Journal of Public Administration Research and Theory online.

We thank Arne Hørlück Høeg for providing excellent research assistance. We are also thankful for the great comments we received from participants at the Administrative Burden pre-conference workshop at the 2022 PMRC.

This work was supported by funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement no. 802244).

No new data were generated or analyzed in support of this research.

Aarøe , Lene , Martin Baekgaard , Julian Christensen , and Donald P. Moynihan . 2021 . Personality and public administration: Policymaker tolerance of administrative burdens in welfare services . Public Administration Review 81 ( 4 ): 652 – 63 .

Google Scholar

Ali , Sameen A. Mohsin , and Samia W. Altaf . 2021 . Citizen trust, administrative capacity and administrative burden in Pakistan’s immunization program . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 17 .

Baekgaard , Martin , Pamela Herd , and Donald P. Moynihan . 2022 . Of “Welfare Queens” and “Poor Carinas” social constructions, deservingness messaging, and the mental health of welfare clients . British Journal of Political Science 53 ( 2 ): 594 – 612 .

Baekgaard , Martin , and Jonas Krogh Madsen . 2023 . Anticipated administrative burden: How proximity to upcoming compulsory meetings affect welfare recipients’ experiences of administrative burden . Public Administration 1 – 19 . doi: 10.1111/padm.12928

Baekgaard , Martin , Kim Sass Mikkelsen , Jonas Krogh Madsen , and Julian Christensen . 2021 . Reducing compliance demands in government benefit programs improves the psychological well-being of target group members . Journal of Public Administration Research and Theory 31 ( 4 ): 806 – 21 .

Baekgaard , Martin , Donald P. Moynihan , and Mette Kjaergaard Thomsen . 2021 . Why do policymakers support administrative burdens? The roles of deservingness, political ideology, and personal experience . Journal of Public Administration Research and Theory 31 ( 1 ): 184 – 200 .

Baekgaard , Martin , and Tara Tankink . 2022 . Administrative burden: Untangling a bowl of conceptual spaghetti . Perspectives on Public Management and Governance 5 ( 1 ): 16 – 21 .

Barnes , Carolyn. nd . Decoupling policy and practice: The redemption costs of WIC . Working Paper.

Google Preview

Barnes , Carolyn , and Virginia Riel . 2022 . “I don’t know nothing about that”: How “learning costs” undermine COVID-related efforts to make SNAP and WIC more accessible . Administration & Society 54 ( 10 ): 1902 – 30 .

Barnes , Carolyn Y. 2021 . “It takes a while to get used to”: The costs of redeeming public benefits . Journal of Public Administration Research and Theory 31 ( 2 ): 295 – 310 .

Barnes , Carolyn Y. , and Julia R. Henly . 2018 . ‘They are underpaid and understaffed’: How clients interpret encounters with street-level bureaucrats . Journal of Public Administration Research and Theory 28 ( 2 ): 456 – 456 .

Bashir , Mohsin , and Muhammed Azfar Nisar . 2020 . Expectation versus reality: Political expediency and implementation of information to information laws . Public Administration Quarterly 44 ( 1 ): 3 – 30 .

Bell , Elizabeth , Julian Christensen , Pamela Herd , and Donald P. Moynihan . 2022 . Health in citizen-state interactions: How physical and mental health problems shape experiences of administrative burden and reduce take-up . The American Review of Public Administration 83 ( 2 ): 385 – 400 .

Bell , Elizabeth , and Katharine Meyer . nd . Can reducing workload enhance equity at the front-lines? How street-level bureaucrats’ capacity impacts access to burdensome public programs . Working Paper.

Bell , Elizabeth , and Kylie Smith . 2022 . Working within a system of administrative burden: How street-level bureaucrats’ role perceptions shape access to the promise of higher education . Administration & Society 54 ( 2 ): 167 – 211 .

Bell , Elizabeth , Ani Ter-Mkrtchyan , Wesley Wehde , and Kylie Smith . 2020 . Just or unjust? How ideological beliefs shape street-level bureaucrats’ perceptions of administrative burden . Public Administration Review 81 ( 4 ): 610 – 24 .

Bhanot , Syon P. 2021 . Good for you or good for us? A field experiment on motivating citizen behavior change . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 14 .

Bhargava , Saurabh , and Dayanand Manoli . 2015 . Psychological frictions and the incomplete take-up of social benefits: Evidence from an IRS field experiment . American Economic Review 105 ( 11 ): 3489 – 529 .

Bisgaard , Mette . 2023 . Dealing with bureaucracy: measuring citizens’ bureaucratic self-efficacy . International Review of Public Administration 28 ( 1 ): 45 – 63 .

Bozeman , Barry , and Jan Youtie . 2020 . Robotic bureaucracy: Administrative burden and red tape in university research . Public Administration Review 80 ( 1 ): 157 – 62 .

Brodkin , Evelyn Z. , and Malay Majmundar . 2010 . Administrative exclusion: Organizations and the hidden costs of welfare claiming . Journal of Public Administration Research and Theory 20 ( 4 ): 827 – 48 .

Bruch , Sarah K. , Myra Marx Ferree , and Joe Soss . 2010 . From policy to polity: Democracy, paternalism, and the incorporation of disadvantaged citizens . American Sociological Review 75 ( 2 ): 205 – 26 .

Burden , Barry C. , David T. Canon , Kenneth R. Mayer , and Donald P. Moynihan . 2012 . The effect of administrative burden on bureaucratic perception of policies: Evidence from election administration . Public Administration Review 72 ( 5 ): 741 – 51 .

Campbell , Jesse W. , Sanjay K. Pandey , and Lars Arnesen . 2022 . The ontology, origin, and impact of divisive public sector rules: A meta‐narrative review of the red tape and administrative burden literatures . Public Administration Review 83 ( 2 ): 296 – 315 .

Carey , Gemma , Helen Dickinson , Eleanor Malbon , Megan Weier , and Gordon Duff . 2020 . Burdensome administration and its risks: Competing logics in policy implementation . Administration & Society 52 ( 9 ): 1362 – 81 .

Carey , Gemma , Eleanor Malbon , and James Blackwell . 2021 . Administering inequality? The national disability insurance scheme and administrative burdens on individuals . Australian Journal of Public Administration 80 ( 4 ): 854 – 72 .

Cecchini , Mathilde. nd . Into the unknown—The administrative burden of uncertainty in citizen-state interactions . Working Paper.

Christensen , Julian , Lene Aarøe , Martin Baekgaard , Pamela Herd , and Donald P. Moynihan . 2020 . Human capital and administrative burden: The role of cognitive resources in citizen-state interactions . Public Administration Review 80 ( 1 ): 127 – 36 .

Chudnovsky , Mariana , and Rik Peeters . 2021a . A cascade of exclusion: Administrative burdens and access to citizenship in the case of Argentina’s National Identity Document . International Review of Administrative Sciences 88 ( 4 ): 1068 – 85 .

———. 2021b . The unequal distribution of administrative burden: A framework and an illustrative case study for understanding variation in people’s experience of burdens . Social Policy and Administration 55 ( 4 ): 527 – 42 .

Collie , Alex , Luke Sheehan , Ashley McAllister , and Genevieve Grant . 2021 . The learning, compliance, and psychological costs of applying for the disability support pension . Australian Journal of Public Administration 80 ( 4 ): 873 – 90 .

Compton , Mallory E. , Matthew M. Young , Justin B. Bullock , and Robert Greer . 2022 . Administrative errors and race: Can technology mitigate inequitable administrative outcomes ? Journal of Public Administration Research and Theory 33 : 512 – 28 .

Cook , Kay. 2021 . Gender, malice, obligation and the state: Separated mothers’ experiences of administrative burdens with Australia’s child support program . Australian Journal of Public Administration 80 ( 4 ): 912 – 32 .

Currie , Janet. 2006 . The take-up of social benefits . In Public policy and the distribution of income , ed. A. J. Auerbach , D. Card , and J. M. Quigley , 80 – 148 . New York : Russell Sage Foundation .

Daigneault , Pierre Marc , and Christian Macé . 2020 . Program awareness, administrative burden, and non-take-up of Québec’s supplement to the work premium . International Journal of Public Administration 43 ( 6 ): 527 – 39 .

Döring , Matthias. 2021 . How-to bureaucracy: A concept of citizens’ administrative literacy . Administration & Society 53 ( 8 ): 1155 – 77 .

Döring , Matthias , and Jonas Krogh Madsen . 2022 . Mitigating psychological costs—The role of citizens’ administrative literacy and social capital . Public Administration Review 82 ( 4 ): 671 – 81 .

Doughty , Meghan , and Karen J. Baehler . 2020 . “Hostages to compliance”: Towards a reasonableness test for administrative burdens . Perspectives on Public Management and Governance 3 ( 4 ): 273 – 87 .

George , Bert , Sanjay K. Pandey , Bram Steijn , Adelien Decramer , and Mieke Audenaert . 2020 . Red tape, organizational performance and employee outcomes: Meta‐analysis, meta‐regression and research agenda . Public Administration Review 81 ( 4 ): 638 – 51 .

Gilad , Sharon , and Michaela Assouline . 2022 . Citizens’ choice to voice in response to administrative burdens . International Public Management Journal : 1 – 22 . doi: 10.1080/10967494.2022.2072988

Griffiths , Rita. 2021 . Universal credit and automated decision making: A case of the digital tail wagging the policy dog ? Social Policy and Society : 1 – 18 .

Haeder , Simon F. , Steven M. Sylvester , and Timothy Callaghan . 2021 . Lingering legacies: Public attitudes about Medicaid beneficiaries and work requirements . Journal of Health Politics, Policy and Law 46 ( 2 ): 305 – 55 .

Halling , Aske. nd . Taxing language: Do interpreting fees affect healthcare usage? Working Paper.

Halling , Aske , Pamela Herd , and Donald Moynihan . 2022 . How difficult should it be? Evidence of burden tolerance from a nationally representative sample . Public Management Review : 1 – 20 . doi: 10.1080/14719037.2022.2056910

Halling , Aske , and Niels Bjørn Grund Petersen . nd . Burden feedback: When citizens communicate burdens, frontline employees respond evidence from an experiment . Working Paper.

Hattke , Fabian , Judith Hattke , David Hensel , Pamela Herd , Janne Kaluzca , Donald P. Moynihan , and Rick Vogel . nd . Face-to-face with administrative burdens: Physiological measures and behavioral consequences of psychological costs . Working Paper.

Hattke , Fabian , David Hensel , and Janne Kalucza . 2020 . Emotional responses to bureaucratic red tape . Public Administration Review 80 ( 1 ): 53 – 63 .

Heinrich , Carolyn J. 2016 . The bite of administrative burden: A theoretical and empirical investigation . Journal of Public Administration Research and Theory 26 ( 3 ): 403 – 20 .

———. 2018 . Presidential address: “A thousand petty fortresses”: Administrative burden in US immigration policies and its consequences . Journal of Policy Analysis and Management 37 ( 2 ): 211 – 39 .

Heinrich , Carolyn J. , and Robert Brill . 2015 . Stopped in the name of the law: Administrative burden and its implications for cash transfer program effectiveness . World Development 72 ( August ): 277 – 95 .

Heinrich , Carolyn J. , Sayil Camacho , Kaitlin Binsted , and Shadlan Gale . 2022 . An audit test evaluation of state practices for supporting access to and promoting Covid-19 vaccinations . Social Science and Medicine 301 : 114880 .

Herd , Pamela , Thomas DeLeire , Hope Harvey , and Donald P. Moynihan . 2013 . Shifting administrative burden to the state: The case of Medicaid take-up . Public Administration Review 73 ( s1 ): s69 – 81 .

Herd , Pamela , and Donald P. Moynihan . 2018 . Administrative burden: Policymaking by other means , 1st ed. New York : Russell Sage Foundation .

Hock , Heinrich , John T. Jones , Michael Levere , and David Wittenburg . 2021 . Using behavioral outreach to counteract administrative burden and encourage take-up of simplified disability payment rules . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 15 .

Holler , Roni , and Noam Tarshish . 2022 . Administrative burden in citizen-state encounters: The role of waiting, communication breakdowns and administrative errors . Social Policy and Society : 1 – 18 .

Hoynes , Hilary , Diane W. Schanzenbach , and Douglas Almond . 2016 . Long-run impacts of childhood access to the safety net . American Economic Review 106 ( 4 ): 903 – 34 .

Jakobsen , Morten , Oliver James , Donald P. Moynihan , and Tina Nabatchi . 2016 . JPART virtual issue on citizen-state interactions in public administration research . Journal of Public Administration Research and Theory 29 ( 4 ): e8 – 15 .

Jenkins , Jade Marcus , and Tutrang Nguyen . 2022 . Keeping kids in care: Reducing administrative burden in state child care development fund policy . Journal of Public Administration Research and Theory 32 ( 1 ): 23 – 40 .

Jilke , Sebastian , Wouter Van Dooren , and Sabine Rys . 2018 . Discrimination and administrative burden in public service markets: Does a public-private difference exist ? Journal of Public Administration Research and Theory 28 ( 3 ): 423 – 39 .

Johnson , Donavon , and Alexander Kroll . 2021 . What makes us tolerant of administrative burden? Race, representation, and identity . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 9 .

Kaufmann , Wesley , Alex Ingrams , and Daan Jacobs . 2021 . Being consistent matters: Experimental evidence on the effect of rule consistency on citizen red tape . American Review of Public Administration 51 ( 1 ): 28 – 39 .

Keiser , Lael R. , and Susan M. Miller . 2020 . Does administrative burden influence public support for government programs? Evidence from a survey experiment . Public Administration Review 80 ( 1 ): 137 – 50 .

Kyle , Michael Anne , and Austin B. Frakt . 2021 . Patient administrative burden in the US health care system . Health Services Research 56 ( 5 ): 755 – 65 .

Larsson , Karl Kristian. 2021 . Digitization or equality: When government automation covers some, but not all citizens . Government Information Quarterly 38 ( 1 ): 101547 – 10 .

Lasky-Fink , Jessica , and Linos . Elizabeth . 2023 . Improving delivery of the social safety net: The role of stigma . Journal of Public Administration Research and Theory .

Linos , Elizabeth , Lisa T. Quan , and Elspeth Kirkman . 2020 . Nudging early reduces administrative burden: Three field experiments to improve code enforcement . Journal of Policy Analysis and Management 39 ( 1 ): 243 – 65 .

Linos , Elizabeth , Vikash Reddy , and Jesse Rothstein . 2022 . Demystifying college costs: How nudges can and can’t help . Behavioral Public Policy : 1 – 22 .

Linos , Elizabeth , and Nefara Riesch . 2020 . Thick red tape and the thin blue line: A field study on reducing administrative burden in police recruitment . Public Administration Review 80 ( 1 ): 92 – 103 .

Linos , Katerina , Melissa Carlson , Laura Jakli , Nadia Dalma , Isabelle Cohen , Afroditi Veloudaki , and Stavros Nikiforos Spyrellis . 2022 . How do disadvantaged groups seek information about public services? A randomized controlled trial of communication technologies . Public Administration Review 82 ( 4 ): 708 – 20 .

Lipsky , Michael. 1980 . Street-level bureaucracy: Dilemmas of the individual in public services . New York : Russell Sage Foundation .

Lopoo , Leonard M. , Colleen Heflin , and Joseph Boskovski . 2020 . Testing behavioral interventions designed to improve on-time SNAP recertification . Journal of Behavioral Public Administration 3 ( 2 ): 1 – 8 .

Madsen , Christian Østergaard , Ida Lindgren , and Ulf Melin . 2022 . The accidental caseworker—How digital self-service influences citizens’ administrative burden . Government Information Quarterly 39 ( 1 ): 101653 .

Madsen , Jonas Krogh. nda . Compliant but discouraged? How administrative burden influence unemployment benefit recipients’ job search motivation . Working Paper.

———. ndb . Frictions on both sides of the counter? A study of red tape among street-level bureaucrats and administrative burden among their clients . Working Paper.

Madsen , Jonas Krogh , Martin Baekgaard , and Jon Kvist . 2022 . Scarcity and the mindsets of social welfare recipients: Evidence from a field experiment . Journal of Public Administration Research and Theory 33 : 675 – 87 .

Madsen , Jonas Krogh , and Kim Sass Mikkelsen . 2022 . How salient administrative burden affects job seekers’ locus of control and responsibility attribution: Evidence from a survey experiment . International Public Management Journal 25 ( 2 ): 241 – 60 .

Madsen , Jonas Krogh , Kim Sass Mikkelsen , and Donald P. Moynihan . 2022 . Burdens, sludge, ordeals, red tape, oh my! A user’s guide to the study of frictions . Public Administration 100 ( 2 ): 375 – 93 .

Masood , Ayesha , and Muhammed Azfar Nisar . 2021 . Administrative capital and citizens’ responses to administrative burden . Journal of Public Administration Research and Theory 31 ( 1 ): 56 – 72 .

Mikkelsen , Kim Sass , Jonas Krogh Madsen , and Martin Baekgaard . 2023 . Is stress among street level bureaucrats associated with experiences of administrative burden among clients? A multilevel study of the Danish unemployment sector . Public Administration Review 1 – 13 . doi: 10.1111/puar.13673

Moynihan , Donald , Eric Giannella , Pamela Herd , and Julie Sutherland . 2022 . Matching to categories: Learning and compliance costs in administrative processes . Journal of Public Administration Research and Theory 32 ( 4 ): 750 – 64 .

Moynihan , Donald , and Joe Soss . 2014 . Policy feedback and the politics of administration . Public Administration Review 74 ( 3 ): 320 – 32 .

Moynihan , Donald P. , Pamela Herd , and Hope Harvey . 2015 . Administrative burden: Learning, psychological, and compliance costs in citizen-state interactions . Journal of Public Administration Research and Theory 25 ( 1 ): 43 – 69 .

Moynihan , Donald P. , Pamela Herd , and Elizabeth Ribgy . 2016 . Policymaking by other means: Do states use administrative barriers to limit access to Medicaid . Administration & Society 48 ( 4 ): 497 – 524 .

Newman , Mark E. J. 2003 . Mixing patterns in networks . Physical Review 67 ( 026126 ): 1 – 13 .

Nicholson-Crotty , Jill , Susan M. Miller , and Lael R. Keiser . 2021 . Administrative burden, social construction, and public support for government programs . Journal of Behavioral Public Administration 4 ( 1 ): 1 – 29 .

Nisar , Muhammad A. 2018 . Children of a lesser god: Administrative burden and social equity in citizen-state interactions . Journal of Public Administration Research and Theory 28 ( 1 ): 104 – 19 .

Nisar , Muhammed A. , and Ayesha Masood . nd . Governance by artifacts: Theory and evidence on materiality of administrative burdens . Working Paper.

Olsen , Asmus Leth. 2017 . Human interest or hard numbers? Experiments on citizens’ selection, exposure, and recall of performance information . Public Administration Review 77 ( 3 ): 408 – 20 .

Olsen , Asmus Leth , Jonas Høgh Kyhse-Andersen , and Donald Moynihan . 2020 . The unequal distribution of opportunity: A national audit study of bureaucratic discrimination in primary school access . American Journal of Political Science , Early view version, 66 ( 3 ): 587 – 603 .

Page , Matthew J. , Joanne E. McKenzie , Patrick M. Bossuyt , Isabelle Boutron , Tammy C. Hoffmann , Cynthia D. Mulrow , Larissa Shamseer , et al.  2021 . The PRISMA 2020 statement: An updated guideline for reporting systematic reviews . BMJ 372 : n71 .

Peeters , Rik. 2020 . The political economy of administrative burdens: A theoretical framework for analyzing the organizational origins of administrative burden . Administration & Society 52 ( 4 ): 566 – 92 .

Peeters , Rik , and Sergio A. Campos . 2021 . Taking the bite out of administrative burdens: How beneficiaries of a Mexican social program ease administrative burdens in street-level interactions . Governance 34 ( 4 ): 1001 – 18 .

Peeters , Rik , Humberto Trujillo Jimenez , Elizabeth O’Connor , Pascual Ogarrio Rojas , Michele Gonzalez Galindo , and Daniela Morales Tenorio . 2018 . Low-trust bureaucracy: Understanding the Mexican bureaucratic experience . Public Administration and Development 38 ( 2 ): 65 – 74 .

Peeters , Rik , César Renteria , and Guillermo M. Cejudo . nd . How information capacity shapes administrative burdens: A comparison of the citizen experience of the COVID-19 vaccination programs in the United States, Mexico, and the Netherlands . Working Paper.

Peeters , Rik , and Arjan Widlak . 2018 . The digital cage: Administrative exclusion through information architecture—The case of the Dutch civil registry’s master data management system . Government Information Quarterly 35 : 175 – 83 .

——— . 2023 . Administrative exclusion in the infrastructure-level bureaucracy: The case of the Dutch daycare benefit scandal . Public Administration Review 83 ( 4 ): 863 – 877 . doi: 10.1111/puar.13615

Perianes-Rodriguez , Antonio , Ludo Waltman , and Nees Jan van Eck . 2016 . Constructing bibliometric networks: A comparison between full and fractional counting . Journal of Informetrics 10 ( 4 ): 1178 – 95 .

Petersen , Ole Helby , Jesper Rosenberg Hansen , and Kurt Houlberg . 2022 . The administrative burden of doing business with the government: Learning and compliance costs in Business-Government interactions . Public Administration Review 1 – 19 . doi: 10.1111/padm.12904

Schneider , Anne , and Helen Ingram . 1993 . Social construction of target populations: Implications for politics and policy . American Political Science Review 87 ( 2 ): 334 – 47 .

Selin , Jennifer L. 2019 . The best laid plans: How administrative burden complicates voting rights restoration . Missouri Law Review 84 ( 4 ): 1 – 38 .

Sievert , Martin , and Jonas Bruder . 2023 . Unpacking the effects of burdensome state actions on citizens’ policy perceptions . Public Administration 1 – 21 .

Sievert , Martin , Dominik Vogel , and Mary K. Feeney . 2020 . Formalization and administrative burden as obstacles to employee recruitment: Consequences for the public sector . Review of Public Personnel Administration 42 ( 1 ): 3 – 30 .

Soss , Joe. 1999 . Lessons of welfare: Policy design, political learning, and political action . American Political Science Review 93 ( 2 ): 363 – 80 .

Soss , Joe , Richard Fording , and Sanford F. Schram . 2011 . The organization of discipline: From performance management to perversity and punishment . Journal of Public Administration Research and Theory 21 : i203 – 32 .

Thomsen , Mette Kjaergaard , Martin Baekgaard , and Ulrich Thy Jensen . 2020 . The psychological costs of citizen coproduction . Journal of Public Administration Research and Theory 30 ( 4 ): 656 – 73 .

Widlak , Arjan , and Rik Peeters . 2020 . Administrative errors and the burden of correction and consequence: How information technology exacerbates the consequences of bureaucratic mistakes for citizens . International Journal of Electronic Governance 12 ( 1 ): 1 – 56 .

Yates , Sophie , Gemma Carey , Eleanor Malbon , and Jen Hargrave . 2022 . “Faceless monster, secret society”: Women’s experiences navigating the administrative burden of Australia’s National Disability Insurance Scheme . Health and Social Care in the Community 50 ( 5 ): e2308 – 17 .

As the only exception, we excluded Herd and Moynihan (2018) from the review. The main points in this book have been covered in several journal articles by the authors and including it would therefore introduce the risk of double-counting arguments.

Many of the working papers were later published. The initial number of working papers was 30.

Since this feedback effect is mainly inspired by policy feedback research, for the sake of simplicity we chose not to show this as an independent arrow in the model.

Supplementary data

Email alerts, citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1477-9803
• Print ISSN 1053-1858
• Copyright © 2024 Public Management Research Association
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Impacts of Artificial Intelligence on Public Administration: A Systematic Literature Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Title: Evaluation of blockchain technology in the three dimensions of sustainability and sustainable development goals: a systematic literature review

Authors : Giovana Catussi Paschoalotto; Paulo Sérgio de Arruda Ignácio; Muriel de Oliveira Gavira

Addresses : School of Applied Sciences, Unicamp University, Pedro Zaccaria St., 1300 – UL97-2, Limeira, 13484-350, Brazil ' School of Applied Sciences, Unicamp University, Pedro Zaccaria St., 1300 – UL97-2, Limeira, 13484-350, Brazil ' School of Applied Sciences, Unicamp University, Pedro Zaccaria St., 1300 – UL97-2, Limeira, 13484-350, Brazil

Abstract : It is argued that technologies of the Industry 4.0 can impact on sustainable development. This paper aims to evaluate blockchain technologies in order to understand their impact on the dimensions of sustainability and drive in which marks of the sustainable development goals it has the capacity to contribute to its achievement. A systematic literature review was carried out and it was found that there is a predominance of applications on the economic and environmental dimensions in private sector, while social results are largely linked to public administration challenges. It was found that blockchain can support the achievement of sustainable development goals, with the exception of poverty, quality education, gender equality and partnerships for the goals, which were not found in the literature. Finally, there are some barriers to the implementation of blockchain as a support tool for sustainability such as low dissemination and high cost, energy consumption and processing time.

Keywords : sustainability; environmental; economic; social; blockchain; sustainable development goals; systematic literature review.

DOI : 10.1504/IJISD.2024.138166

International Journal of Innovation and Sustainable Development, 2024 Vol.18 No.3, pp.227 - 256

Received: 09 Jun 2021 Accepted: 07 Mar 2022 Published online: 30 Apr 2024 *

Keep up-to-date

• Our Newsletter ( subscribe for free )
• New issue alerts
• Inderscience is a member of publishing organisations including:

Posterior reversible encephalopathy syndrome associated with antibiotic therapy: a case report and systematic review

• Review Article
• Open access
• Published: 29 April 2024

Cite this article

You have full access to this open access article

• Lorenzo Barba 1 ,
• Carmelo Carrubba 1 ,
• Kai Spindler 2 ,
• Christopher M. Weise 1 ,
• Torben Sachs 3 ,
• Matteo Foschi 4 ,
• Lucio D’Anna 5 , 6 ,
• Bernhard Sehm 1 ,
• Richard Ibe 1 ,
• Erck Elolf 3 ,
• Christian Strauss 2 ,
• Markus Otto 1 ,
• Alexander Mensch 1 &
• Samir Abu-Rumeileh   ORCID: orcid.org/0000-0003-0631-8506 1  

117 Accesses

5 Altmetric

Explore all metrics

Posterior reversible encephalopathy syndrome (PRES) is an acute neurological condition associated with different etiologies, including antibiotic therapy. To date, most data regarding antibiotic-related PRES are limited to case reports and small case series. Here, we report a novel case description and provide a systematic review of the clinico-radiological characteristics and prognosis of available cases of PRES associated with antibiotic therapy. We performed a systematic literature search in PubMed and Scopus from inception to 10 January 2024, following PRISMA guidelines and a predefined protocol. The database search yielded 12 subjects (including our case). We described the case of a 55-year-old female patient with PRES occurring one day after administration of metronidazole and showing elevated serum neurofilament light chain protein levels and favorable outcome. In our systematic review, antibiotic-associated PRES was more frequent in female patients (83.3%). Metronidazole and fluoroquinolones were the most reported antibiotics (33.3% each). Clinical and radiological features were comparable to those of PRES due to other causes. Regarding the prognosis, about one third of the cases were admitted to the intensive care unit, but almost all subjects (90.0%) had a complete or almost complete clinical and radiological recovery after prompt cessation of the causative drug. Antibiotic-associated PRES appears to share most of the characteristics of classic PRES. Given the overall good prognosis of the disease, it is important to promptly diagnose antibiotic-associated PRES and discontinue the causative drug.

Avoid common mistakes on your manuscript.

Introduction

Posterior reversible encephalopathy syndrome (PRES) is an acute neurological syndrome associated with subcortical vasogenic oedema, particularly in the posterior parieto-occipital regions. Typical clinical manifestations include non-specific symptoms such as headache and dizziness, but also more severe features such as altered consciousness, visual impairment and seizures [ 1 , 2 ]. The pathophysiology of PRES is not fully understood yet but appears to be related to dysregulation of cerebral blood flow, cytokine release, disruption of the blood–brain barrier. This may result in endothelial damage and fluid leakage [ 2 ]. The posterior regions of the brain appear to be particularly susceptible to hyperperfusion and vasogenic oedema because of the relatively reduced sympathetic innervation in the posterior fossa, and consequently lower autoregulatory potential [ 1 ]. Indeed, in the majority of cases (approximately 75–80%) patients diagnosed with PRES showed moderate to severe increase of blood pressure (BP) values [ 3 ]. Other conditions that predispose to PRES include eclampsia/pre-eclampsia and renal failure, but also autoimmune or genetic disorders [ 1 , 4 , 5 , 6 , 7 ]. In addition, the development of PRES may be secondary to the administration of drugs. A recent study based on the World Health Organization (WHO) global pharmacovigilance database identified 152 drugs significantly associated with PRES, most of which were antineoplastic, immunomodulatory, and antimicrobial agents [ 2 ]. These agents might cause injury or inhibition of endothelial cell proliferation [ 2 ] (Fig.  1 ).

Clinical features and pathophysiological mechanisms of PRES associated with antibiotic and other drug administration. Disruption of the blood–brain barrier (BBB) with endothelial dysfunction (red box) leads to fluid leakage and vasogenic oedema with or without haemorrhage (blue box) in the central nervous system (CNS) interstitium. Drug intake (e.g., antineoplastic, immunomodulatory, and antimicrobial agents) might cause injury or inhibition of endothelial cell proliferation. Arterial hypertension is also a contributor in PRES pathophysiology. Involved risk factors are represented in red squares. Main clinical manifestations associated with PRES are summarized in the light blue box on the left part of the figure

To date, most data on antibiotic-associated PRES are limited to case reports and small case series. However, given the widespread use of antibiotics in daily clinical practice, it is important to better understand this rare but severe complication of antibiotics. This should also be emphasized in the scenario where non-specific symptoms related to PRES in antibiotic-treated patients may be overlooked and mistakenly attributed to underlying infections as well as other comorbidities that complicate hospitalisation. Furthermore, given the overall good prognosis of PRES with appropriate treatment, it is important to promptly identify potential predisposing factors and discontinue the agent in drug-associated PRES.

Considering all these issues, we aimed to describe a novel case of PRES after metronidazole administration and to perform a systematic review of the clinico-radiological characteristics and prognosis of available cases of PRES associated with antibiotic therapy.

Case description

A 55-year-old woman with no history of immunological, renal or cardiovascular disease was admitted to our neurosurgery department for planned surgical removal of a right vestibular schwannoma causing long-standing right-sided hypoacusis. After the complete removal of the mass, a right-sided peripheral facial nerve palsy (Grade 4, according to the House-Brackmann score) [ 8 ] was noted as a postoperative complication. This was treated with eye protection and physiotherapy.

The patient received no antithrombotic or anticoagulant medication before and after surgery.

Two days after successful surgery, the patient complained of abdominal pain and diarrhea. Microbiological testing revealed positive stool samples for Clostridium difficile , for which metronidazole (400 mg × 3/day) was administered. One day after starting metronidazole, the patient developed a diffuse mild headache (numerical rating scale 4/10), dizziness and nausea, followed by altered mental status with somnolence. Neurological examination revealed left-sided homonymous hemianopia without further focal deficits. BP was modestly elevated at clinical onset (158/60 mmHg). Computed tomography (CT) revealed a right occipital intracerebral haemorrhage with peripheral oedema, right subdural and subarachnoid bleeding (Fig.  2 A) as well as contralateral grey-white matter junction oedema. Laboratory parameters indicated a mild inflammatory reaction (Supplementary Table  1 ). Renal function was unremarkable [glomerular filtration rate, GFR > 90 ml/min (normal range > 60), creatinine 62 micromol/L (normal range 44–809) and electrolytes [Na + 144 mmol/l (normal range 136–145), K + 4.0 mmol/l (normal range 3.4–4.5), Ca ++ 2.19 mmol/l (normal range 2.15–2.50), Cl − 107 (normal range 98–107)]. The patient was therefore admitted to the neurological intensive care unit (ICU) and the arterial hypertension was treated with continuous administration of urapidil and nimodipine. Over the next days, BP values remained normal without the need for intravenous antihypertensive medication. Headache was treated with metamizole and fentanyl transdermal patch. Magnetic resonance imaging (MRI), performed one day after symptom onset, showed subcortical T2, fluid-attenuated inversion recovery (FLAIR) and apparent diffusion coefficient (ADC) hyperintensities in occipital areas (Fig.  2 B-C) without diffusion restriction signals reflecting vasogenic oedema (Supplementary Fig.  1 ). MRI including T2* gradient-echo sequences did not reveal findings associated with cerebral amyloid angiopathy [ 9 ]. Based on clinical and radiological findings, the diagnosis of PRES was made [ 1 ]. Given the previous association between PRES and metronidazole administration reported in the literature [ 10 , 11 , 12 ], metronidazole was discontinued 3 days after first administration and replaced by vancomycin (125 mg × 4/die). Neurofilament light chain protein (NfL) was measured in serum using the ELLA technology as previously described [ 13 ]. Serum NfL was massively elevated two days after symptom onset (166 pg/ml, normal values < 35 pg/ml according to our in-house reference values). Within 2 days of discontinuing metronidazole, the patient showed rapid clinical improvement with reduction of headache and improvement of mental status. At follow-up (4 days after symptom onset) serum NfL concentration were further increased (433 pg/ml).

Imaging findings in our case of PRES after metronidazole administration. Native brain CT imaging showed A) right-sided occipital intracerebral haemorrhage with peripheral oedema (thin arrow), subdural bleeding (large arrow); left-sided grey-white matter junction oedema (*). MRI performed 2 days after clinical onset revealed bilateral occipital B) FLAIR (thin arrow and *) and C) ADC hyperintensities (thin arrow and *) indicating vasogenic edema. The relative right occipital hypointensity on the ADC map is related to the haemorrhage. D) Follow-up MRI after 2 months from clinical onset showed in the ADC map almost complete resolution of the vasogenic oedema and subtotal resorption of parenchymal haemorrhage (thin arrow and *). Abbrevations. ADC: apparent diffusion coefficient; CT: computed tomography; FLAIR: fluid-attenuated inversion recovery; MRI: magnetic resonance imaging; PRES: posterior reversible encephalopathy syndrome

After progressive improvement, the patient was discharged 13 days after the onset of PRES without hemianopia or other neurological sequelae, except for the unchanged right facial palsy. Follow-up MRI at approximately 2 months showed almost complete radiological recovery with resolution of the vasogenic oedema and subtotal resorption of parenchymal haemorrhage (Fig.  2 D).

We conducted a systematic study-level meta-analysis according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and a protocol agreed upon by all authors. Four authors (LB, SAR, LD, MF) searched for articles in publicly available literature databases (PubMed, Scopus) from inception to 10th January 2024. The search strategy was based on the combination of antibiotic names listed in the World Health Organization (WHO) Aware, Watch and Reserve (AWaRe) Classification 2021 [ 14 ] and "pres" or "posterior reversible encephalopathy syndrome" as either keywords or Medical Subject Heading (MeSH) terms (full search string is provided in the Supplementary materials ). Reference lists and citing articles were also reviewed to increase the identification rate of relevant studies. Titles and abstracts were screened independently. Potentially relevant articles were acquired in full text and assessed for eligibility by the same four authors working in pairs. The final selection was shared among all the four authors. Disagreements were resolved by consensus.

Non-original records, studies with no full text available and articles in languages other than English were excluded. In addition, we included only studies that reported: 1) diagnosis of PRES [ 1 , 2 ]; 2) recent administration of antibiotics and medical indication thereof; 3) information on symptoms and vital parameters at clinical onset; 4) at least one neuroimaging marker supporting the diagnosis of PRES [ 1 , 2 ].

From the included studies, we extrapolated the following data, if available: 1) general information and clinical history of the patient, including all prescribed drugs and corresponding medical indications; 2) symptoms at clinical onset and after drug withdrawal; 3) vital and laboratory parameters; 4) therapeutic management of PRES; 5) neuroimaging findings at clinical onset and follow-up. The strength of the association between antibiotic prescription and PRES symptoms was assessed through the Naranjo Adverse Drug Reaction Probability Scale (NADRPS) [ 15 ].

By using the reported search string (Supplementary materials, Appendix ), we identified a total of 232 articles published in literature. After duplicate removal, 187 articles were screened. Title and abstract screening led to the selection of 24 articles for full read, which brought us to a final number of 11 articles fulfilling the inclusion criteria and reporting one case each (Supplementary Table  1 ) [ 10 , 11 , 12 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Full data on literature case reports (n = 11) and on our case are summarized in Table  1 and detailed in Supplementary Table  1 .

Demographic and clinical characteristics of reported cases

We performed the final analysis on 12 subjects, including our case. Patients had a mean age of 50.2 years (± standard deviation: 22.3 years; range: 16–87) and were predominantly female (10 of 12, 83.3%). Patients had multiple comorbidities, especially cardiovascular, renal and metabolic diseases (Table  1 ). The list of all prescribed antibiotics, their indications and other clinical information regarding the cases are reported in detail in Supplementary Table  1 and are summarized in Table  1 . No pathogen-specific patterns were found, but gastrointestinal infections were most common (4 of 12, 33.3%). Of note, three articles [ 10 , 11 , 18 ] described the possible association of 2 antibiotics each with PRES. Metronidazole and fluoroquinolones were the most commonly reported antibiotics, each being administered in 4 cases (33.3%) (Table  1 ). The association between antibiotic administration and PRES, as assessed by the NADRPS score, was considered possible in 11 cases including our case report (NADRPS score 1–4, 91.7%) and probable in 1 case (NADRPS score 5, 8.3%). The types of infection and, when reported, the underlying pathogens were heterogeneous between studies and did not show any specific pattern (Table  1 and Supplementary Table  2 ).

Clinical, laboratory and radiological findings at clinical onset

PRES manifestations started within one week of antibiotic administration in most cases (7 of 12 reports, 66.7%) and, most commonly, included altered mental status (11 of 12 reports, 91.7%), seizures (7 of 12 studies, 58.3%) and headache (6 of 12 reports, 50.0%), often in combination (Table  1 ). At presentation, patients often had elevated BP (> 140/90 mmHg) (6 of 12 reports, 50.0%), but PRES also occurred in the setting of normal (90–140/60–90 mmHg) (5 of 12 reports, 41.7%) or decreased BP (< 90/60 mmHg) (1 of 12 reports, 8.3%). Laboratory changes related to the ongoing inflammatory process were found in 7 of 12 reports (58.3%), while electrolyte disturbances and renal dysfunction were described in 4 (33.3%) and 5 of 12 reports (41.7%), respectively. In addition, pathological electroencephalogram (EEG) findings were  reported in 4 out of 12 subjects (33.3%). In all cases, MRI performed during the acute phase showed posterior hyperintensities in occipital areas (11 of 12 reports, 91.7%) or the pons (1 of 12, 8.3%), compatible with PRES. Our case was the only one to show lobar haemorrhage occurring together with bilateral vasogenic oedema. ICU admission and mechanical ventilation were required in 4 of 12 (33.3%) and 2 of 12 (16.7%) cases, respectively. Antiepileptic therapy was administered in 5 of 12 patients (41.7%) (Table  1 ).

Follow-up data

The vast majority of subjects showed significant clinical improvement within a few days after antibiotic cessation, with no neurological sequelae at follow-up (9 out of 10 cases reported, 90.0%, in 2 articles no follow-up data were reported). Furthermore, follow-up imaging showed complete radiological recovery in almost all cases (9 out of 10 reported cases, 90.0%, in 2 articles no follow-up data were reported) (Table  1 ).

In this study, we described a case of PRES associated with recent antibiotic therapy and reviewed similar reports in the literature. Our main findings suggest that PRES associated with antibiotic therapy: 1) can occur at any age and may affect both sexes, with a higher frequency in female patients; 2) is often, but not necessarily, associated with arterial hypertension; 3) occurs within a few days of drug administration, presents with altered mental status and/or seizures, and shows pathological MRI findings in the parieto-occipital areas, as well as heterogeneous EEG changes; 4) may require admission to ICU and mechanical ventilation in about one third of cases, but in the vast majority of patients show complete or almost complete clinical and radiological recovery after prompt discontinuation of the drug; 5) may not be antibiotic or pathogen specific, although metronidazole and fluoroquinolones were the most commonly reported antibiotics.

Overall, our findings are in line with previous literature on the pathophysiological and clinical features of PRES; indeed, previous studies reported also on the female predominance in drug-associated PRES cases [ 2 ]. Further, the elevation of arterial BP is thought to be the driving pathological mechanism in most cases of PRES, leading to cerebral hyper-perfusion [ 1 ]. In our study, the frequencies of patients with hyper-, normo- and hypotension well reflects those of previous reports (i.e., approximately 30–50% of PRES cases without elevated BP values) [ 3 ]. In addition, altered mental status (92%), headache (50%) and seizures (58%) were common findings in the patient group analyzed, largely reflecting the prevalences reported in other PRES cohorts [ 1 , 5 , 24 , 25 , 26 ]. To date, there are no data to support specific antiepileptic treatment regimens for PRES compared with those used in other conditions. Furthermore, although seizures in PRES may be associated with a worse clinical outcome [ 26 ] in most cases they do not recur [ 27 ]. Prospective studies are therefore urgently needed to assess the impact of antiepileptic therapy on outcomes in PRES.

Brain MRI is currently the gold standard for the diagnosis of PRES [ 1 ]. Although parieto-occipital hyperintensities are the classic neuroradiological features of PRES [ 1 ], concomitant intraparenchymal or subarachnoid haemorrhages, like in our case report, have been described in literature in patients with PRES not associated with antibiotics [ 24 , 28 , 29 , 30 , 31 ]. Our reported case is therefore peculiar because it is the first to be described. As previously hypothesised, reperfusion injury due to vasoconstriction or pial rupture secondary to hypertension appears to be the leading pathogenic mechanism in PRES-associated intracranial haemorrhage [ 24 ].

NfL is a marker of neuroaxonal damage in primary and non-primary neurological diseases [ 32 ]. To date, only one study has investigated and demonstrated an increase in serum NfL in patients with PRES [ 33 ]. Our case report supports these observations, but we cannot exclude the significant confounding effects of cerebral haemorrhage and neurosurgical treatment on NfL concentrations as described previously [ 13 , 34 ].

Regarding the prognosis, most cases of PRES associated with antibiotic therapy or other drugs showed a complete or almost complete clinical recovery [ 35 ], however also permanent disability and death have been reported in few cases [ 36 ].

Of note, metronidazole and fluoroquinolones turned out to be the antibiotics more frequently associated with PRES in our systematic review. Some authors have hypothesized that metronidazole treatment may lead to autonomic dysfunction and, thus, to PRES. Indeed, a case of metronidazole-induced sympathetic autonomic dysfunction has been described [ 37 ]. Interestingly, metronidazole therapy has previously been related also to another neurological syndrome, namely metronidazole-induced encephalopathy (MIE). Indeed, MIE can present with encephalopathic symptoms as well as seizures mimicking PRES [ 38 ]. However, unlike PRES, MIE often occurs weeks/months after drug ingestion, presenting primarily with motor symptoms (e.g., dysarthria, postural instability, and oculomotor dysfunction) due to the involvement of cerebellar and brainstem structures [ 38 ]. In addition, in MIE, MRI shows characteristic reversible symmetrical T2/FLAIR hyperintense lesions of the dentate nuclei in the vast majority of patients [ 38 ]. Taking in to account these findings, we suggest that a misdiagnosis might possibly explain the low number of PRES cases associated with metronidazole prescription in studies of pharmacovigilance databases [ 38 ].

With regard to fluoroquinolones, these drugs are known to cause central and peripheral neurotoxicity with variousmanifestations such as headache and seizures [ 39 ], but the underlying mechanisms of fluoroquinolone-associated PRES remain unclear [ 12 ]. Nevertheless, given the frequent use of these antibiotics and the relatively modest absolute risk of adverse neurological effects following fluoroquinolone administration [ 39 ], it is important to consider PRES early in the differential diagnosis of acute neurological symptoms emerging on this therapy. Furthermore, in clinical practice it can sometimes be difficult to understand whether the antiinfective therapy or a specific infectious disease itself is the trigger of PRES. For example, in a literature case of PRES in a human immunodeficiency virus (HIV)-infected patient on antiretroviral therapy, both the pathogen and the drug were discussed to play a pathogenic role [ 40 ].

The main limitation of our review is its retrospective nature; therefore, our findings should be confirmed by pharmacovigilance investigations, or by prospective or translational studies to identify the pathophysiological mechanisms underlying antibiotic-associated PRES. Moreover, further studies on PRES associated with other classes of drugs, especially antiviral and antineoplastic agents, are needed.

In conclusion, the occurrence of acute or subacute neurological symptoms after antibiotic treatment should raise the clinical suspicion towards PRES. MIE should be considered in the differential diagnosis in case of metronidazole prescription [ 41 ]. Brain MRI allows rapid and accurate identification of vasogenic oedema consistent with PRES, which could otherwise be missed on CT alone. Indeed, in our case description, the peculiar presence of concomitant intracerebral haemorrhage could have been misleading without advanced imaging. Early recognition and appropriate management of PRES is necessary to eliminate any predisposing causes. This would lead to a shorter recovery time, reduced need for intensive care and overall better clinical outcomes.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Fugate JE, Rabinstein AA (2015) Posterior reversible encephalopathy syndrome: clinical and radiological manifestations, pathophysiology, and outstanding questions. Lancet Neurol 14:914–925. https://doi.org/10.1016/S1474-4422(15)00111-8

Article   PubMed   Google Scholar  

Balcerac A, Bihan K, Psimaras D, Lebrun-Vignes B, Salem JE, Weiss N (2023) Drugs associated with posterior reversible encephalopathy syndrome, a worldwide signal detection study. J Neurol 270:975–985. https://doi.org/10.1007/s00415-022-11450-y

Rabinstein AA, Mandrekar J, Merrell R, Kozak OS, Durosaro O, Fugate JE (2012) Blood pressure fluctuations in posterior reversible encephalopathy syndrome. J Stroke Cerebrovasc Dis 21:254–258. https://doi.org/10.1016/j.jstrokecerebrovasdis.2011.03.011

Mueller-Mang C, Mang T, Pirker A, Klein K, Prchla C, Prayer D (2009) Posterior reversible encephalopathy syndrome: do predisposing risk factors make a difference in MRI appearance? Neuroradiology 51:373–383. https://doi.org/10.1007/s00234-009-0504-0

Fugate JE, Claassen DO, Cloft HJ, Kallmes DF, Kozak OS, Rabinstein AA (2010) Posterior reversible encephalopathy syndrome: associated clinical and radiologic findings. Mayo Clin Proc 85:427–432. https://doi.org/10.4065/mcp.2009.0590

Article   PubMed   PubMed Central   Google Scholar  

Shaharir SS, Remli R, Marwan AA, Said MS, Kong NC (2013) Posterior reversible encephalopathy syndrome in systemic lupus erythematosus: pooled analysis of the literature reviews and report of six new cases. Lupus 22:492–496. https://doi.org/10.1177/0961203313478303

Article   CAS   PubMed   Google Scholar  

Cani I, Righini M, Cenni P, Foschi M (2022) Teaching NeuroImage: Partially Reversible Widespread Leukoencephalopathy Associated With Atypical Hemolytic Uremic Syndrome. Neurology 99:1128–1129. https://doi.org/10.1212/WNL.0000000000201378

House JW, Brackmann DE (1985) Facial nerve grading system. Otolaryngol Head Neck Surg 93(2):146–147. https://doi.org/10.1177/019459988509300202

Charidimou A, Boulouis G, Frosch MP, Baron JC, Pasi M, Albucher JF, Banerjee G, Barbato C, Bonneville F, Brandner S, Calviere L, Caparros F, Casolla B, Cordonnier C, Delisle MB, Deramecourt V, Dichgans M, Gokcal E, Herms J, Hernandez-Guillamon M, Jäger HR, Jaunmuktane Z, Linn J, Martinez-Ramirez S, Martínez-Sáez E, Mawrin C, Montaner J, Moulin S, Olivot JM, Piazza F, Puy L, Raposo N, Rodrigues MA, Roeber S, Romero JR, Samarasekera N, Schneider JA, Schreiber S, Schreiber F, Schwall C, Smith C, Szalardy L, Varlet P, Viguier A, Wardlaw JM, Warren A, Wollenweber FA, Zedde M, van Buchem MA, Gurol ME, Viswanathan A, Al-Shahi Salman R, Smith EE, Werring DJ, Greenberg SM (2022) The Boston criteria version 2.0 for cerebral amyloid angiopathy: a multicentre, retrospective. MRI-neuropathology diagnostic accuracy study. Lancet Neurol. 21(8):714–725

PubMed   Google Scholar  

Kikuchi S, Orii F, Maemoto A, Ashida T (2016) Reversible Posterior Leukoencephalopathy Syndrome Associated with Treatment for Acute Exacerbation of Ulcerative Colitis. Intern Med 55:473–477. https://doi.org/10.2169/internalmedicine.55.5250

Gilbert BW, Gabriel A, Velazquez L (2020) A Case of Antibiotic-Induced Posterior Reversible Encephalopathy Syndrome. Hosp Pharm 55:338–341. https://doi.org/10.1177/0018578719844706

Hosaka T, Terada M, Okubo K, Iwabuchi S, Kajii E (2023) Management of a rare case of metronidazole-induced encephalopathy presenting as posterior reversible encephalopathy syndrome: a case report. Neurol Sci 44:1101–1103. https://doi.org/10.1007/s10072-022-06480-3

Abu-Rumeileh S, Barba L, Bache M, Halbgebauer S, Oeckl P, Steinacker P, Güttler A, Keßler J, Illert J, Strauss C, Vordermark D, Otto M (2023) Plasma β-synuclein, GFAP, and neurofilaments in patients with malignant gliomas undergoing surgical and adjuvant therapy. Ann Clin Transl Neurol 10:1924–1930. https://doi.org/10.1002/acn3.51878

Article   CAS   PubMed   PubMed Central   Google Scholar  

World Health Organization (WHO) Aware, Watch and Reserve (AWaRe) Classification 2021 (2021) https://www.who.int/publications/i/item/2021-aware-classification

Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, Janecek E, Domecq C, Greenblatt DJ (1981) A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 30:239–245. https://doi.org/10.1038/clpt.1981.154

Tomar LR, Pandita N, Arya S, Agrawal C (2022) Linezolid-Associated Posterior Reversible Leuco-encephalopathy Syndrome in a Patient with Disseminated Tuberculosis. Trop Doct 52:196–198. https://doi.org/10.1177/00494755211053194

Ali WH. Ciprofloxacin-associated posterior reversible encephalopathy (2013) BMJ Case Rep:bcr2013008636. https://doi.org/10.1136/bcr-2013-008636 .

Bitar De Zayas-Enriquez A, Soper C (2019) Daptomycin-Induced Posterior Reversible Encephalopathy Syndrome (PRES). Case Rep Neurol Med 2019:8756932. https://doi.org/10.1155/2019/8756932

Cordeanu EM, Gaertner S, Faller A, Mirea C, Lessinger JM, Kemmel V, Stephan D (2017) Rifampicin reverses nicardipine effect inducing uncontrolled essential hypertension. Fundam Clin Pharmacol 31:587–589. https://doi.org/10.1111/fcp.12292

Kim L, Kolar L, Abu A, Wu LJ (2021) Posterior Reversible Encephalopathy Syndrome in a Case of Vancomycin-Induced Acute Kidney Injury. J Clin Neurol 17:140–141. https://doi.org/10.3988/jcn.2021.17.1.140

Nagel S, Köhrmann M, Huttner HB, Storch-Hagenlocher B, Schwab S (2007) Linezolid-induced posterior reversible leukoencephalopathy syndrome. Arch Neurol 64:746–748. https://doi.org/10.1001/archneur.64.5.746

Nasu K, Fujisawa M, Kato H, Nangaku M (2017) Three cases of posterior reversible encephalopathy syndrome with chronic kidney disease triggered by infection. Nephrology (Carlton) 22:322–325. https://doi.org/10.1111/nep.12930

Reyes HA, Witsch J, Sueldo C, Hong J (2017) Posterior reversible encephalopathy in an adult patient with poststreptococcal glomerulonephritis. BMJ Case Rep :bcr2017220043. https://doi.org/10.1136/bcr-2017-220043 .

Triplett JD, Kutlubaev MA, Kermode AG, Hardy T (2022) Posterior reversible encephalopathy syndrome (PRES): diagnosis and management. Pract Neurol 22:183–189. https://doi.org/10.1136/practneurol-2021-003194

Liman TG, Bohner G, Heuschmann PU, Endres M, Siebert E (2012) The clinical and radiological spectrum of posterior reversible encephalopathy syndrome: the retrospective Berlin PRES study. J Neurol 259:155–164. https://doi.org/10.1007/s00415-011-6152-4

Bastide L, Legros B, Rampal N, Gilmore EJ, Hirsch LJ, Gaspard N (2018) Clinical Correlates of Periodic Discharges and Nonconvulsive Seizures in Posterior Reversible Encephalopathy Syndrome (PRES). Neurocrit Care 29:481–490. https://doi.org/10.1007/s12028-018-0548-2

Kastrup O, Gerwig M, Frings M, Diener HC (2012) Posterior reversible encephalopathy syndrome (PRES): electroencephalographic findings and seizure patterns. J Neurol 259:1383–1389. https://doi.org/10.1007/s00415-011-6362-9

Cordelli DM, Masetti R, Ricci E, Toni F, Zama D, Maffei M, Gentili A, Parmeggiani A, Pession A, Franzoni E (2014) Life-threatening complications of posterior reversible encephalopathy syndrome in children. Eur J Paediatr Neurol 18:632–640. https://doi.org/10.1016/j.ejpn.2014.04.014

Pilato F, Calandrelli R, Distefano M, Panfili M, Della Marca G, Colosimo C (2019) Acute radiological pattern and outcome in posterior reversible encephalopathy syndrome patients. Clin Neurol Neurosurg 185:105459. https://doi.org/10.1016/j.clineuro.2019.105459

Vallone MG, Vázquez C, Pigretti S, Oses LL, Angriman F, Zurru MC (2020) Clinical Presentation and Risk Factors for Poor Outcomes Among Adult Patients With Posterior Reversible Encephalopathy Syndrome: A Retrospective Cohort Study. Neurologist 25:162–167. https://doi.org/10.1097/NRL.0000000000000294

Appiani F, Claverie CS, Klein FR (2023) Posterior Reversible Leukoencephalopathy With Hemorrhagic Features: A Case Series. Cureus 15(11):e49587. https://doi.org/10.7759/cureus.49587

Abu-Rumeileh S, Abdelhak A, Foschi M, D’Anna L, Russo M, Steinacker P, Kuhle J, Tumani H, Blennow K, Otto M (2023) The multifaceted role of neurofilament light chain protein in non-primary neurological diseases. Brain 146(2):421–437. https://doi.org/10.1093/brain/awac328

Fang X, Liang Y, Zhang W, Wang Q, Chen J, Chen J, Lin Y, Chen Y, Yu L, Wang H, Chen D (2021) Serum Neurofilament Light: a Potential Diagnostic and Prognostic Biomarker in Obstetric Posterior Reversible Encephalopathy Syndrome. Mol Neurobiol 58:6460–6470. https://doi.org/10.1007/s12035-021-02562-z

Zheng P, Wang X, Chen J, Wang X, Shi SX, Shi K (2023) Plasma Neurofilament Light Chain Predicts Mortality and Long-Term Neurological Outcomes in Patients with Intracerebral Hemorrhage. Aging Dis 14(2):560–571

PubMed   PubMed Central   Google Scholar  

Tlemsani C, Mir O, Boudou-Rouquette P, Huillard O, Maley K, Ropert S, Coriat R, Goldwasser F (2011) Posterior reversible encephalopathy syndrome induced by anti-VEGF agents. Target Oncol 6:253–258. https://doi.org/10.1007/s11523-011-0201-x

Legriel S, Schraub O, Azoulay E, Hantson P, Magalhaes E, Coquet I, Bretonniere C, Gilhodes O, Anguel N, Megarbane B, Benayoun L, Schnell D, Plantefeve G, Charpentier J, Argaud L, Mourvillier B, Galbois A, Chalumeau-Lemoine L, Rivoal M, Durand F, Geffroy A, Simon M, Stoclin A, Pallot JL, Arbelot C, Nyunga M, Lesieur O, Troché G, Bruneel F, Cordoliani YS, Bedos JP, Pico F, Critically III Posterior Reversible Encephalopathy Syndrome Study Group (CYPRESS) (2012) Determinants of recovery from severe posterior reversible encephalopathy syndrome. PLoS One 7:e44534. https://doi.org/10.1371/journal.pone.0044534

Hobson-Webb LD, Roach ES, Donofrio PD (2006) Metronidazole: newly recognized cause of autonomic neuropathy. J Child Neurol 21(5):429–431. https://doi.org/10.1177/08830738060210051201

Sørensen CG, Karlsson WK, Amin FM, Lindelof M (2020) Metronidazole-induced encephalopathy: a systematic review. J Neurol 267:1–13. https://doi.org/10.1007/s00415-018-9147-6

Ellis DE, Hubbard RA, Willis AW, Zuppa AF, Zaoutis TE, Hennessy S (2021) Comparative neurological safety of fluoroquinolones versus therapeutic alternatives. Pharmacoepidemiol Drug Saf 30(6):797–805. https://doi.org/10.1002/pds.5219

Birner B, Hirzel C, Wagner F, Waldegg G (2018) Posterior reversible encephalopathy syndrome in an HIV-infected patient on antiretroviral treatment: what is the risk factor? BMJ Case Rep 2018:bcr2017221998. https://doi.org/10.1136/bcr-2017-221998

Bhattacharyya S, Darby RR, Raibagkar P, Gonzalez Castro LN, Berkowitz AL (2016) Antibiotic-associated encephalopathy. Neurology 86:963–971. https://doi.org/10.1212/WNL.0000000000002455

Download references

Acknowledgements

We are indebted to the staff of the Neurology Laboratory of Halle University Hospital. LB received research support from the Medical Faculty of Martin-Luther-University Halle-Wittenberg (Junior Clinician Scientist Programm No. JCS24/02). SAR received research support from the Medical Faculty of Martin-Luther-University Halle-Wittenberg (Clinician ScientistProgramm No. CS22/06).

Open Access funding enabled and organized by Projekt DEAL. No funding was received for conducting this study.

Author information

Authors and affiliations.

Department of Neurology, Martin-Luther-University Halle-Wittenberg, Halle (Saale), Germany

Lorenzo Barba, Carmelo Carrubba, Christopher M. Weise, Bernhard Sehm, Richard Ibe, Markus Otto, Alexander Mensch & Samir Abu-Rumeileh

Department of Neurosurgery, Martin-Luther-University Halle-Wittenberg, Halle (Saale), Germany

Kai Spindler & Christian Strauss

Department of Radiology, Martin-Luther-University Halle-Wittenberg, Halle (Saale), Germany

Torben Sachs & Erck Elolf

Department of Biotechnological and Applied Clinical Sciences, University of L’Aquila, L’Aquila, Italy

Matteo Foschi

Department of Stroke and Neuroscience, Charing Cross Hospital, Imperial College London NHS Healthcare Trust, London, UK

Lucio D’Anna

Division of Brain Sciences, Department of Medicine, Hammersmith Campus, Imperial College London, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

Lorenzo Barba, Carmelo Carrubba, Alexander Mensch and Samir Abu-Rumeileh contributed to the study conception and design. All authors performed the literature search and/or data analysis. The first draft of the manuscript was written by Lorenzo Barba, Carmelo Carrubba, Alexander Mensch and Samir Abu-Rumeileh and all authors commented on previous versions of the manuscript. All authors critically revised the final manuscript.

Corresponding author

Correspondence to Samir Abu-Rumeileh .

Ethics declarations

Research involving human participants and/or animals.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This is a case report including one patient and a literature review. No ethical approval is required.

Disclosure of potential conflicts of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Informed consent

The patient included in this study provided informed consent for the publication of anonymized data in accordance with the declaration of Helsinki.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Alexander Mensch and Samir Abu-Rumeileh shared last co-authorship.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 1062 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Barba, L., Carrubba, C., Spindler, K. et al. Posterior reversible encephalopathy syndrome associated with antibiotic therapy: a case report and systematic review. Neurol Sci (2024). https://doi.org/10.1007/s10072-024-07545-1

Download citation

Received : 01 March 2024

Accepted : 20 April 2024

Published : 29 April 2024

DOI : https://doi.org/10.1007/s10072-024-07545-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Posterior reversible encephalopathy syndrome
• Antibiotics
• Adverse drug reactions
• Metronidazole
• Neurofilament light chain
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 30.4.2024 in Vol 10 (2024)

COVID-19 Vaccine Hesitancy: Umbrella Review of Systematic Reviews and Meta-Analysis

Authors of this article:

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Best practices for the dissemination and implementation of neuromuscular training injury prevention warm-ups in youth team sport: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0009-0005-0529-0398 Destiny Lutz 1 ,
• http://orcid.org/0000-0001-6429-4333 Carla van den Berg 1 ,
• http://orcid.org/0000-0003-3056-8169 Anu M Räisänen 1 , 2 ,
• Isla J Shill 1 , 3 ,
• Jemma Kim 4 , 5 ,
• Kenzie Vaandering 1 ,
• Alix Hayden 6 ,
• http://orcid.org/0000-0002-0427-2877 Kati Pasanen 1 , 7 , 8 , 9 ,
• http://orcid.org/0000-0002-5951-5899 Kathryn J Schneider 1 , 3 , 8 , 9 , 10 ,
• http://orcid.org/0000-0002-9499-6691 Carolyn A Emery 1 , 3 , 8 , 9 , 11 , 12 , 13 ,
• http://orcid.org/0000-0002-5984-9821 Oluwatoyosi B A Owoeye 1 , 4
• 1 Sport Injury Prevention Research Centre, Faculty of Kinesiology , University of Calgary , Calgary , Alberta , Canada
• 2 Department of Physical Therapy Education - Oregon , Western University of Health Sciences College of Health Sciences - Northwest , Lebanon , Oregon , USA
• 3 Hotchkiss Brain Institute , University of Calgary , Calgary , Alberta , Canada
• 4 Department of Physical Therapy & Athletic Training , Doisy College of Health Sciences, Saint Louis University , Saint Louis , Missouri , USA
• 5 Interdisciplinary Program in Biomechanics and Movement Science , University of Delaware College of Health Sciences , Newark , Delaware , USA
• 6 Libraries and Cultural Resources , University of Calgary , Calgary , Alberta , Canada
• 7 Tampere Research Center for Sports Medicine , Ukk Instituutti , Tampere , Finland
• 8 McCaig Institute for Bone and Joint Health , University of Calgary , Calgary , Alberta , Canada
• 9 Alberta Chilrden's Hopsital Research Institute , University of Calgary , Calgary , Alberta , Canada
• 10 Sport Medicine Centre , University of Calgary , Calgary , Alberta , Canada
• 11 O'Brien Institute for Public Health , University of Calgary , Calgary , Alberta , Canada
• 12 Department of Community Health Sciences , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• 13 Department of Paediatrics , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• Correspondence to Ms Destiny Lutz, Sport Injury Prevention Research Centre, Faculty of Kinesiology, University of Calgary, Calgary, Alberta, Canada; destiny.lutz{at}ucalgary.ca

Objective To evaluate best practices for neuromuscular training (NMT) injury prevention warm-up programme dissemination and implementation (D&I) in youth team sports, including characteristics, contextual predictors and D&I strategy effectiveness.

Design Systematic review.

Data sources Seven databases were searched.

Eligibility The literature search followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Inclusion criteria: participation in a team sport, ≥70% youth participants (<19 years), D&I outcomes with/without NMT-related D&I strategies. The risk of bias was assessed using the Downs & Black checklist.

Results Of 8334 identified papers, 68 were included. Sport participants included boys, girls and coaches. Top sports were soccer, basketball and rugby. Study designs included randomised controlled trials (RCTs) (29.4%), cross-sectional (23.5%) and quasi-experimental studies (13.2%). The median Downs & Black score was 14/33. Injury prevention effectiveness (vs efficacy) was rarely (8.3%) prioritised across the RCTs evaluating NMT programmes. Two RCTs (2.9%) used Type 2/3 hybrid approaches to investigate D&I strategies. 19 studies (31.6%) used D&I frameworks/models. Top barriers were time restrictions, lack of buy-in/support and limited benefit awareness. Top facilitators were comprehensive workshops and resource accessibility. Common D&I strategies included Workshops with supplementary Resources (WR; n=24) and Workshops with Resources plus in-season Personnel support (WRP; n=14). WR (70%) and WRP (64%) were similar in potential D&I effect. WR and WRP had similar injury reduction (36–72%) with higher adherence showing greater effectiveness.

Conclusions Workshops including supplementary resources supported the success of NMT programme implementation, however, few studies examined effectiveness. High-quality D&I studies are needed to optimise the translation of NMT programmes into routine practice in youth sport.

Data availability statement

Data are available in a public, open access repository. Not Applicable.

https://doi.org/10.1136/bjsports-2023-106906

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THE TOPIC

Neuromuscular training (NMT) injury prevention warm-up programmes are effective at preventing injury rates in youth sports. However, for proper dissemination and implementation (D&I) by multiple stakeholders, barriers such as low adoption, adherence and lack of time must be addressed.

WHAT THIS STUDY ADDS

There are limited high-quality research studies to facilitate the widespread adoption of, and improved adherence to, NMT programmes. Few studies used D&I theories, frameworks or models. Programme flexibility is a common barrier to implementation; adaptation of NMT programmes to fit local contexts is imperative. Comprehensive workshops and supplementary resources currently support the success of NMT programme implementation.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Promotion of NMT programmes as the standard of practice is essential to increase practical D&I of these programmes, and thus reduce the burden of youth sport injuries. This work provides some directions for stakeholders, including researchers, implementation support practitioners and youth sport policymakers, on current best practices for the delivery of NMT programmes in local youth sport settings. This work also provides the evidence base for more translational research efforts in youth sport injury prevention, a much-needed next step to optimise NMT programmes into youth sport practice.

Introduction

Youth (<19 years) sport participation provides numerous benefits, positively impacting physical and mental health. 1 Youth sport participation rates are high, with up to 90% of youth participating in sport globally. 2–5 However, with increased sport participation comes increased injury risk. One-in-three youth sustain a sport-related injury each year, leading to a significant public health burden with high healthcare costs. 3 6–8 Sport-related injuries may also result in long-term health consequences (eg, poor mental health, reduced physical activity, post-traumatic osteoarthritis). 7–9 Implementing injury prevention strategies is critical to mitigate the injury risk associated with youth sport participation.

Neuromuscular training (NMT) injury prevention warm-up programmes in youth team sport are effective in reducing injury rates by up to 60% and decreasing costs associated with injury based on randomised controlled trials (RCT) and systematic reviews. 10–21 NMT programmes include exercises that can be categorised across aerobic, balance, strength and agility components 22 23 and typically take 10–15 min. 24 25 Originally implemented with the intention of reducing non-contact lower extremity injury risk, 26–28 the effectiveness of NMT programmes has since been evaluated across numerous sports, age groups and levels of play and are associated with lower extremity and overall injury rates compared with standard of practice warm-ups. 12 20 21 25 In youth team sports, a protective effect has been demonstrated in soccer, handball, basketball, netball, rugby and floorball. 11 16 29–31 The International Olympic Committee Consensus Statement on Youth Athletic Development recommends multifaceted NMT warm-up programmes in youth sport. 32

Despite being a primary injury prevention strategy across youth sports, NMT programme adoption remains low. 33–38 For evidence-informed interventions to be successful and have a practical impact, pragmatic approaches derived from dissemination and implementation (D&I) science are necessary across multiple socioecological levels including organisation, coach and player. 36 Dissemination is defined as ‘the active process of spreading evidence-based interventions to a target population through determined channels and using planned strategies’. Implementation is ‘the active process of using strategies across multiple levels of change to translate evidence-based interventions into practice and prompt corresponding behaviour change in a target population’. 36

The aim of this systematic review was to evaluate current best practices for the D&I of NMT programmes in a youth team sport. The specific objectives of this systematic review were to: (1) describe the characteristics of identified D&I-related studies (studies with at least one D&I outcome directly or indirectly assessed as primary, secondary or tertiary outcome); (2) evaluate factors associated with the D&I of NMT warm-up programme across socioecological levels, including barriers and facilitators; (3) examine the effect of D&I strategies in delivering NMT warm-ups across multiple socioecological levels; and (4) examine the influence of D&I strategies on injury rates. Our protocol was registered in PROSPERO (CRD42021271734), and the review is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( supplemental appendix S1 ).

Supplemental material

Search strategy and data sources.

A comprehensive search was developed with a librarian (KAH) in MEDLINE, incorporating four main concepts: child/youth, injury prevention, implementation/compliance/adherence and sports. The author team reviewed the final search strategy which was then piloted against the known key studies to ensure that the search was capturing relevant studies. Finally, the MEDLINE search was translated to the other databases. Searches were conducted 25 August 2021 (updated 16–18 August 2022; 5 September 2023). Search strategies are available in Supplemental Appendix S2 . Studies were identified by searching seven databases: MEDLINE(R) and EPUB Ahead of Print, In-Process & Other Non-Indexed Citations and Daily, Embase, Cochrane Central Register of Controlled Trials, Cochrane database of Systematic Reviews (all Ovid); CINAHL Plus with Full Text, SPORTDiscus with Full Text (EBSCO) and ProQuest Dissertations & Thesis Global.

Study selection and eligibility

All database search results were uploaded and duplicates were removed in Covidence (Veritas Health Innovation, Melbourne, Australia). Records were independently reviewed by authors in pairs (DL/IJS, CV/JK, KV/DL), starting with a screening of 50 randomly selected citations to assess inter-rater agreement with a threshold set at 90%. Each pair of reviewers performed title/abstract screening and full-text screening independently, providing reasons for exclusion at full-text stage ( figure 1 ). Any disagreements for exclusion, where a consensus could not be reached within pairs, were resolved by a senior author (OBAO). A secondary evaluation of included manuscripts was performed by senior authors (OBAO and CAE) to ensure appropriate inclusion. Study inclusion criteria were: (1) Participation in a team sport (male and female); (2) a minimum of 70% of participants as a youth (<19 years) or coaches of these youth teams; (3) reported dissemination and/or implementation outcomes (eg, self-efficacy, adherence, intention); (4) reported D&I strategies related to NMT warm-up programmes (ie, NMT delivery strategies, where applicable eg, in RCTs). Exclusion criteria were: (1) Studies evaluating rehabilitation programmes, non-team-based or physical education programmes; (2) non-peer-reviewed; (3) not English. The screening process was reported using the PRISMA flow diagram. 39

• Download figure
• Open in new tab
• Download powerpoint

Study identification Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.

Risk of bias

To assess the risk of bias, three sets of paired reviewers independently used the Downs & Black (D&B) quality assessment tool. 40 The tool consists of a 27-item checklist (total score/33). A third senior reviewer (OBAO or CAE or AMR) resolved any disagreements. The rating of evidence and strength of recommendations were assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines. 41–43

Frameworks/models

The proportion of studies that used D&I research theories/frameworks/models, including behaviour change frameworks/models, was examined to identify commonly used frameworks/models.

Efficacy-effectiveness orientation in RCTs

We assessed the components of 12 RCTs using the Rating Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool, as adapted by Maddox et al 44 RITES scores RCTs in systematic reviews based on a continuum of efficacy-effectiveness across four domains: Participant characteristics, trial setting, flexibility of intervention(s) and clinical relevance of experimental and comparison intervention(s) ( online supplemental table S1 ). 45 We modified the Likert grading system to classify studies depending on whether their emphasis was more on efficacy or effectiveness or balanced for both. Given that different aspects of each trial may fall in different places along the efficacy-effectiveness continuum, each RITES domain is scored independently and a composite score is not applicable. To minimise subjectivity, the RITES evaluation for included RCTs was completed by two reviewers (AMR and OBAO). Any disagreements were resolved through discussion to reach a consensus.

Study typologies and assessment of study relevance to D&I

The level of relevance of individual studies (RCTs and quasi-experiments) to D&I was determined based on the implementation-effectiveness hybrid taxonomy: Type 1 (primarily focused on clinical/intervention outcomes), Type 2 (balance focused on both clinical/intervention outcomes and D&I outcomes) and Type 3 (primarily or ‘fully’ (our adaptation) focused on D&I outcomes) studies. 36 46 For ease of interpretation of results, studies were rated considering three broad traditional research design categories (ie, hierarchy of evidence): RCTs, quasi-experimental and observational studies, including cohort, cross-sectional, pre-experimental, qualitative, mixed-methods and ecological studies. Observational studies were categorised as ‘fully focused’ observational-implementation (if only D&I outcomes were evaluated) or ‘partially focused’ observational-implementation (if a combination of clinical and D&I outcomes were evaluated’ D&I studies. 47 RCTs and quasi-experimental studies with Type 2 or Type 3 hybrid approaches were indicated as ‘highly relevant’ towards informing D&I best practices. Furthermore, observational-implementation studies that are fully focused on D&I were also indicated as ‘highly relevant’.

Data extraction

The extracted data included: study design, author, journal, year, population (eg, 13–17 years old female soccer players), participant demographics, D&I intervention strategies (eg, workshops, supplementary resources), D&I framework/model, control group strategies, D&I outcomes (eg, adoption, adherence, intention, fidelity, self-efficacy) and injury outcomes. Study design classification was completed based on data extracted and the process taken by authors, 48 which may have differed from the original classification. Furthermore, prospective, and retrospective cohort studies were consolidated into ‘cohort’ to improve ease of readability. D&I outcomes indicated as compliance were included in the appropriate adherence category as defined in Owoeye et al and described as ‘adherence-related’ outcomes, to maintain unified language across results; the full list is provided in online supplemental table S2 . 36 49–51 Based on the dose-response thresholds reported for NMT programmes within current literature, measures of adherence were used to indicate potential D&I effect ( online supplemental table S5 ). 24 36 52–54 Studies with cumulative utilisation (sessions completed/total possible) of ≥70%, utilisation frequency of ≥1.5 sessions/week or a significant association between D&I exposures and outcomes were defined as moderate-to-highly relevant and identified as having a potential D&I effect (ie, yes). Studies presenting cumulative utilisation <70%, utilisation frequency <1.5 sessions/week or no association between D&I exposures and outcomes were defined as low-to-no relevance (ie, no). Studies reporting both frequency utilisation and cumulative utilisation must both reach the established dose-response thresholds to be considered as having a potential D&I effect. D&I barriers and facilitators, factors influencing injury prevention implementation success and the identification of any frameworks used were also extracted and categorised into themes. Measures of potential effect for these results were summarised using OR, proportions and mean differences in D&I outcomes (eg, adoption, adherence). Injury-specific results were reported as incidence rate ratios, risk ratios, ORs or prevalence. D&I strategies were classified into various categories, including workshops, supplementary resources, personnel support, supervision and combinations of these strategies.

Equity, diversity and inclusion statement

Our author team is comprised of student and senior researchers across various disciplines with representation from low-to-middle-income countries. A variety of demographic, socioeconomic and cultural backgrounds were included in our study populations.

PRISMA flow, characteristics of included studies and risk of bias assessment

A total of 68 relevant studies were included from our initial and updated search yield of 9021 studies ( figure 1 ). Across included studies, 13 included only male youth participants, 13 included only female youth participants, 26 included both and 16 reported coach-focused findings. Sports represented were soccer (n=33), rugby (n=8), basketball (n=7), multisport (n=7), handball (n=5), floorball (n=3), field hockey (n=3), volleyball (n=1) and futsal (n=1).

Details of study characteristics and risk of bias are presented in online supplemental table S3 . D&B scores ranged from 4/33 to 24/33 (median=14/33) from a variety of study designs, including 20 RCTs, 16 cross-sectional, 9 quasi-experimental, 8 cohort, 6 qualitative, 3 ecological, 3 mixed-methods and 3 pre-experimental. The D&B scores for the two top D&I-related relevant studies—an RCT Type 2 study (n=1) was 21/33 and an RCT Type 3 hybrid study (n=1) was 17/33. Using the GRADE guidelines for the process of rating the quality of evidence available and interpreting the quality assessment, the strength of recommendations was ‘low’ given the multiplicity of designs. 42 43

Characteristics of current D&I-related studies

23 studies (33.8%) reported using a D&I /behaviour change framework/model. D&I frameworks included Reach, Effectiveness, Adoption, Implementation and Maintentance (RE-AIM) Framework (n=7), Consolidation Framework for Implementation Research (n=1), Precede-Proceed Model (n=1), Translating Research into Injury Prevention Practice (n=1) and Promoting Action on Research Implementation of Health Sciences (n=1) and the Adherence Optimisation Framework (n=1). Behaviour change models included the Health Action Process Approach (HAPA) (n=8), Theory of Planned Behaviour (n=1) and the Health Belief Model (n=1).

Assessment of study relevance to D&I

Two RCTs of 68 included D&I-related studies (2.9%) were identified as highly relevant to D&I best practices (ie, Type 2 or 3 hybrid approach). 55 56 18 (27.9%) RCTs reported a secondary analysis of D&I strategies 12 16 19 30 53 57–69 ; classified as Type 1 hybrids. Five (8.3%) quasi-experimental studies used Type 2 or Type 3 hybrid approach 22 70–73 ; the remaining studies (n=4; 5%) were classified as quasi-experimental Type 1 hybrids. 74–77 Many observational studies (n=17; 26.7%), 78–94 were highly relevant based on being fully-focused observational-implementation studies; 5 (6.7%) were partially-focused observational-implementation studies. 52 95–98 The remaining observational studies (n=17; 23.3%) were observational-implementation studies, 35 99–114 reporting D&I outcomes from a qualitative lens using interviews and surveys.

The RITES scores for the 14 D&I-related RCTs that examined injuries as primary outcome and D&I outcomes as secondary (Type 1 hybrid approach) are presented in table 1 . Almost all (13 of 14; 92.9%) of the RCTs focused mainly on intervention efficacy (as opposed to effectiveness) regarding the flexibility of NMT warm-up programmes. Cumulatively, effectiveness was rarely (7.1%) prioritised as a primary focus across all the 56 possible ratings of the RITES domains for all 14 studies. 50% of the domain ratings demonstrated efficacy as a priority and 42.9% of the ratings were indicated for a balance between efficacy and effectiveness.

• View inline

RITES domain scores for included trials

Contextual predictors of NMT warm-up programme implementation

50 (73.5% of total) studies identified ≥1 barrier or facilitator within their findings, with 10 (14.7% of total) specifically examining barriers/facilitators as their main objectives. Full list is provided in online supplemental table S4 . The most common barriers identified were time restrictions (n=30), 30 35 59 62 69 70 73 74 78 79 81 82 84 87–91 93 96 98 101 102 105 107 108 112 114 reduced buy-in/support (n=8) 62 75 84 87 105 110–112 and limited awareness of preventative effects of programmes (n=8). 74 84 103 104 107 109 113 Facilitators included comprehensive workshops from trained instructors (n=11), 53 71 78–80 84 90 96 99 100 112 accessibility of supplementary resources (n=10) 82 84 87 89 90 105 114 and uptake/support from multiple stakeholders (n=7). 56 67 84 101 103 105 112 Moreover, suggestions from multiple socioecological levels indicated that increasing programme education and support, increased sport-specific activities and improved awareness of preventive effects, influence NMT implementation success. 36 88 89 115 116 Figure 2 , adapted from Basow et al 117 illustrates the contextual factors reported in the literature. 117 This evidence-informed model shows the important barriers and facilitators that influence the end-user implementation of NMT warm-up programmes across the three key socioecological levels of change.

Contextual predictors of NMT implementation across multiple socioecological levels. (Adapted from Basow et al (2021)).(116). Notes. SE, self-efficacy. NMT, neuromuscular training. Bold represents top barrier(s)/facilitator(s).

51 (75%) studies used implementation strategies for NMT warm-up programmes. The most frequently used strategies were Workshops with supplementary Resources (WR; n=24), followed by Workshops with supplementary Resources, plus in-season Personnel support (WRP; n=14). Three studies employed both WR and WRP strategies. Other methods for implementation included only workshops (n=9), only supplementary resources (n=4), supplementary resources and personnel support (n=2), workshops with personnel support (n=1) and supervision (n=1). Note, some studies are duplicated throughout the table when multiple D&I strategies are compared. 22 53 56 86

The key D&I concepts that were reported within the included studies were adherence or adherence-related (eg, self-efficacy, translation and perception). Specific outcomes within these concepts were further examined from the individual study results. We did not have enough evidence to present a meta-analysis of the effect of D&I strategies on D&I outcomes. Therefore, online supplemental table S5 presents a qualitative summary of the relationships between reported D&I exposure and D&I outcomes. 40 studies reported adherence-related outcomes, of which 32 (80%) were indicated to have potential D&I effect. Studies using WRP (n=14) reported completing between 1.4 and 2.6 sessions/week and cumulative utilisation of 39–85.6%; 9 of these 14 studies have potential D&I effect. Studies using WR (n=24) presented utilisation frequency ranging from 0.8 to 3.2 sessions/week and cumulative utilisation of 55–98% of sessions; 16 of these 24 have potential D&I effect. In studies evaluating workshops only (n=9; 22%), frequency utilisation was reported between 1 and 2 sessions/week across eight of the nine studies and one study had 52% cumulative utilisation; two have potential D&I effect.

Effects of D&I strategies on injury outcomes

Three RCTs specifically examined the effects of the D&I strategies used to deliver NMT programmes on injury outcomes ( table 2 ). Two studies that compared both WR and WRP to supplementary resource only found no significant differences between strategies, 53 56 they reported reduced injury rates in the highest adherence groups by 56% and 72%, respectively. Another study comparing WR and WRP to a standard of practice warm-up found a 36% reduction of ankle and knee injuries when using WR and a 38% reduction in ankle and knee injuries without supervision. 22 There were no significant differences in injury rates between groups.

Injury Outcomes by D&I strategies and adherence

This study evaluated current literature to inform evidence-based best practices for the D&I of NMT programmes in youth team sport. To our knowledge, this is the first systematic review evaluating the D&I of NMT programmes in youth sport. To improve the practical implementation of NMT warm-ups, factors associated with implementation success and current best practices for delivering context-specific NMT programmes are required to be evaluated. 118 In this review, we found few D&I-related studies use D&I or behaviour change frameworks, theories or models to guide their research questions. We discovered the number of RCTs examining the effectiveness of D&I strategies for NMT programme delivery is limited. Common barriers to NMT implementation include programme flexibility and time restrictions; and the use of coach workshops and supplementary resources are currently the primary strategy in NMT programme D&I facilitation.

One-third of the included studies used a D&I framework or behaviour change model in their research work. The HAPA and RE-AIM models were the most frequently used. These models are a conceptual and organised combination of theories required to direct the design, evaluation and translation of evidence-based interventions (NMT programmes) and the context in which they are being implemented. 36 71 119 It is imperative for D&I studies to use these frameworks/models to fully understand specific implementation processes and contexts. Future D&I studies should consider using appropriate frameworks or models, including adaptations and combination of models to guide their specific aims.

Relevance to D&I

Across the relevant literature, a variety of designs and levels of evidence were included.

Of 68 studies, 7 (10.3%) were found to be ‘highly relevant’ toward informing D&I best practice (2 (2.9%) RCTs, 5 (7.4%) quasi-experimental). Other ‘relevant’ studies evaluated implementation as secondary objectives (Type 1 hybrid designs) and/or were of lower level of evidence. 33 observational studies were ‘highly relevant’ to D&I, assessing D&I outcomes and barriers and facilitators from a qualitative lens. While these studies are important for understanding D&I context, more high-quality and highly relevant studies such as RCTs and quasi-experimental designs using the Type 3 hybrid approach, or non-hybrid approach focused on solely evaluating the effectiveness of D&I strategies, are needed to advance the widespread adoption and continued use of NMT programmes in youth team sport.

Effectiveness versus efficacy

Effectiveness is indicative of an evidence-informed intervention’s readiness for practical implementation. 36 Findings from our RITES scores evaluation indicate that the majority of the RCTs had a primary focus on efficacy and not effectiveness. Although many RCT studies had a fair balance between efficacy and effectiveness for participant characteristics, trial settings and clinical relevance domains (≥50% of RCTs), there is a lack of flexibility in the development and evaluation of the evidence supporting current NMT warm-up programmes. These disparities regarding practical implementation have implications for D&I research and practice in this field. Current NMT programmes may need to be modified or adapted to the local context and evaluated further to improve implementation in youth sport settings.

Contextual considerations

In our Adapted Socioecological Model ( figure 2 ), we demonstrate that the utilisation of NMT programmes by individual players within youth team sport can depend on their coach adopting and implementing the warm-up, which may also be dependent on larger organisational systems. Barriers related to end-users’ success in wide-spread adoption and long-term maintenance can be moderated; however, researchers and implementers have to be intentional about tackling these recognised barriers and associated challenges 25 87 104 115 ; integrating the facilitators of successful implementation intending to reduce and address these obstacles is essential. The barriers and facilitators identified in this systematic review provide insight into the combination of D&I strategies that should be formulated and tested by D&I researchers in the sports injury prevention field.

Within the current review, lack of time, whether it be learning, instructing and/or practicing the programme, is a common barrier that plays a significant role in implementation. A recent narrative review focused solely on the barriers and facilitators associated with exercise-based warm-up programmes showed similar conclusions regarding time restrictions. 115 Collective themes within this literature for players, coaches and organisations found that reduced buy-in and support at different levels impacted the adoption of NMT warm-up programmes. The lack of awareness and knowledge of the injury prevention benefits of NMT warm-up programmes also presented major barriers to buy-in, leading to reduced implementation success. Future interventions should ensure that education about evidence-informed injury prevention outcomes associated with programme adherence is integrated within their D&I strategies.

D&I science is a growing field of study. A variety of D&I outcomes were identified such as self-efficacy, intention, reach, outcome expectancy and most commonly, adherence or adherence-related outcomes. These outcomes were evaluated using different D&I strategies for NMT warm-up programmes. The most commonly reported strategies were Workshops with supplementary Resources with/without in-season Personnel support. Evaluation of D&I outcomes showed that adherence or adherence-related outcomes were most frequently reported across studies. Various measures of adherence as defined by Owoeye et al (2020) were identified, including cumulative utilisation, utilisation frequency, utilisation fidelity, duration fidelity and exercise fidelity. 36

Adherence remains the most common D&I outcome in the sport injury prevention literature. 36 120 In this review, we defined adherence and adherence-related thresholds for a moderate-to-high dose-response to be ≥70% cumulative utilisation and/or ≥1.5 session/week to achieve the desired protective effects. This was done with consideration of pragmatism and a practical balance between programme efficacy and effectiveness given the existing literature. 24 91 32 of 40 studies (80%) from those with adherence or adherence-related outcomes had a potential D&I effect based on a moderate-to-high adherence or adherence-related outcome level. The use of WR and WRP was the most common D&I strategies for delivering NMT warm-up programmes. While there are several areas for improvement for the practical D&I of NMT warm-up programmes in youth sport settings, the use of comprehensive workshops and supplementary resources at various levels, particularly with coaches, appears to be the optimal best practice. However, only two ‘highly relevant’ D&I studies (RCTs) from the current systematic review presented conclusions based on the effectiveness of D&I strategies and outcomes specifically.

Many studies (n=26/68; 38.2%) included both male and female participants; however, no sex-differences were described. When examining D&I outcomes, only 7/26 (26.9%) had moderate-to-high adherence when looking at both male and female youth players. In total, 84.6% of the female-only (11/13) and 72.7% of the male-only studies (8/11) reported moderate-to-high adherence levels. These findings suggest greater attention and efforts for adherence and implementation of NMT programmes in the male youth team sport setting may need further consideration compared with the female youth sport context.

Of the preliminary evidence for Type 2 and 3 hybrid designs, the literature highlighted in the synthesis of this data that WR are effective strategies in injury prevention and showed more moderate-to-high adherence levels. Given that most studies are doing some form of WR, adding in-season personnel support does not increase the protective effect and may be less sustainable given that resources, time and support are significant barriers to the D&I of these programmes.

Additionally, greater implementation and programme buy-in were found in studies where uptake of these NMT programmes was supported across multiple stakeholders, particularly at the organisation level. 19 67 90 103 112 Catering to programme deliverers (coaches, organisations, parents) and evaluating their awareness, perception and self-efficacy may help further inform our understanding of D&I and how we can best work to promote programme uptake further.

D&I strategies and injury outcomes

The findings from this systematic review suggest that while various D&I intervention strategies are effective at reducing injuries in youth team sports, the ranges of injury rate ratios are similar across studies employing different strategies (32–88% lower injury rates across WR strategy studies and 41–77% lower injury rates across WRP strategy studies). 22 53 56 Although this was not the proposed evaluation of these studies, our findings demonstrate that the use of workshops may influence D&I success and the availability of supplementary resources alone may not be efficacious. Future evaluation of the influence of delivery strategies should be considered.

Future directions

Using facilitators to reduce barrier burden.

Regarding NMT strategy evaluation, our findings show that most of the current programmes focus on efficacy over effectiveness, particularly in the aspect of intervention flexibility; this suggests a need for the adaptation of NMT programmes to fit local contexts. NMT programme developers should consider more enjoyable and user-friendly exercises that include sport-specific activities (eg, ball work, partner drills, tags). Increasing variations also improves player buy-in and increases intrinsic motivation. At a coaching level, workshops on NMT programmes should include evidence-informed education on the injury prevention benefits and should incorporate content addressing coach self-efficacy to enhance implementation quality. 16 100 121 An ongoing pragmatic evaluation of NMT programme effectiveness is warranted as they undergo adaptation to local contexts.

Organisations have expressed limited knowledge and education for implementation as a significant barrier to successful NMT programme use. 90 99 101 105 112 115 122 Implementers should look to provide accessible resources and encourage further support from multiple stakeholders, including the governing bodies. This could lead to policy changes within the club and result in greater uptake of these programmes long-term. Collaborations among stakeholders (researchers, youth sport administrators, coaches and players) in programme development, evaluation, D&I are necessary to improve efforts for impactful practical translation of programmes.

Research recommendations

The support for NMT programmes within youth sport is extensive. 28 Although these programmes have been shown to be effective for injury prevention in many sports, 10 11 sport representation across D&I studies in our review was limited. Scaling up of NMT programmes and supporting continued research into other sports is vital for increased context-specific D&I of these programmes to reduce the overall burden of youth sport injuries.

Compliance and adherence were often used interchangeably, despite having distinct definitions. Although their mathematical calculations are similar, these two constructs are contextually different. Compliance refers to individuals conforming to prescribed recommendations in controlled intervention settings, 123 while adherence refers to the agreement of an individual’s behaviour to recommended evidence-based interventions in uncontrolled settings. 36 Standardised definitions should be considered more frequently by researchers to build on current knowledge and inform future D&I research.

Using D&I frameworks/models can improve NMT programme implementation success in a practical setting. 71 124 Application of D&I frameworks/models, including behaviour change models, 124 is limited in injury prevention and this is reflected in the current systematic review. Future studies should use D&I frameworks/models to help guide the implementation of these NMT programmes. In doing so, researchers can gain a better understanding of the contextual and behaviour change aspects related to youth sport injury prevention. 115

Limitations

Given the broad nature of our research question, specific results were required for inclusion. Despite being specific to our objectives, our limitations set for participant age range, team sport settings and English language studies only, may have resulted in missing other studies that evaluated D&I interventions and outcomes related to NMT programmes.

Due to the heterogeneous nature of studies, meta-analysis was not possible for any of our objectives. Inclusion of various study designs, although comprehensive, impeded this process and resulted in inconsistent injury and adherence definitions across our population of interest. Furthermore, the subjective nature of many qualitative studies included may have resulted in variability within the data extracted. With the varied definitions used for each specific outcome and design, we looked to consolidate the terminology used into more succinct and unified language and we encourage this to be employed by researchers.

Methodological flaws existed in the included studies that may warrant caution about the interpretation of our conclusions. For example, many of the included studies did not include power calculations or reported low power, increasing the chance of Type 2 error. Further, many studies did not consider confounding or effect modification in their analyses or failed to report the validity of measurement tools used for injury data collection. We also acknowledge that publication bias may have favoured the inclusion of studies demonstrating significant findings (eg, effectiveness, efficacy). By considering quality assessment as an objective, we aimed to account for these limitations.

There was limited evidence supporting the effect of D&I strategies on D&I-specific outcomes. There were only two high-level evidence (RCTs) studies in this review that directly discussed the matter of D&I strategies on D&I outcomes. 55 56 D&I-related outcomes were evaluated as secondary objectives in other high-level evidence studies, therefore, we could only examine the relationship between D&I strategy and outcome to assess if the strategy used resulted in moderate-to-high adherence levels, given our pre-established thresholds.

Conclusions

This systematic review demonstrates that: (1) Few D&I-related studies are based on D&I or behaviour change theories, frameworks or models; (2) few RCTs have examined the effectiveness of D&I strategies for delivering NMT programmes; (3) programme flexibility and time restrictions are the most common barriers to implementation and; (4) a combination of coach workshops and supplementary resources are currently the primary strategy facilitating NMT programme D&I; however their effectiveness is only evaluated in a few studies. This systematic review provides foundational evidence to facilitate evidence-informed knowledge translation practices in youth sport injury prevention. Transitioning to more high-quality D&I research RCTs and quasi-experimental designs that leverage current knowledge of barriers and facilitators, incorporates Type 2 or Type 3 hybrid approaches and uses behaviour change frameworks are important next steps to optimise the translation of NMT programmes into routine practice in youth team sport settings.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

• Cumming SP ,
• Australian Sports Commission
• Meeuwisse DW ,
• Eliason PH , et al
• Child and Adolescent Health Measurement Initiative
• Emmonds S ,
• Weaving D ,
• Lara-Bercial S , et al
• Merkel DL ,
• Raysmith BP ,
• Charlton PC
• Brunner R ,
• Friesenbichler B ,
• Casartelli NC , et al
• Whittaker JL , et al
• Meeuwisse WH
• Gianotti S ,
• Jacquet C ,
• Verhagen E , et al
• Marshall DA ,
• Lopatina E ,
• Lacny S , et al
• Owoeye OBA ,
• Akinbo SRA ,
• Tella BA , et al
• Palacios-Derflingher LM ,
• Pasanen K ,
• Parkkari J ,
• Pasanen M , et al
• Soligard T ,
• Myklebust G ,
• Steffen K , et al
• Thorborg K ,
• Krommes KK ,
• Esteve E , et al
• Webster KE ,
• Räisänen AM , et al
• Räisänen AM ,
• van den Berg C ,
• Owoeye OBA , et al
• Pfeifer K , et al
• Richmond SA ,
• Doyle-Baker PK , et al
• Barengo N ,
• Meneses-Echávez J ,
• Ramírez-Vélez R , et al
• Bizzini M ,
• Sadigursky D ,
• De Lira DNL , et al
• Åkerlund I ,
• Sonesson S , et al
• Hislop MD ,
• Stokes KA ,
• Williams S , et al
• Hopper AJ ,
• Joyce C , et al
• Bergeron MF ,
• Mountjoy M ,
• Armstrong N , et al
• De Ste Croix M ,
• Sanchez SH , et al
• Creech MJ ,
• Peterson DC , et al
• Rauvola RS ,
• Brownson RC
• Owoeye OB ,
• Olawale OA , et al
• Niederer D ,
• Vogt L , et al
• Ardern CL ,
• Büttner F ,
• Andrade R , et al
• Guyatt GH ,
• Vist GE , et al
• Kunz R , et al
• Siemieniuk R ,
• Maddox CD ,
• Subialka JA ,
• Young JL , et al
• Wieland LS ,
• Berman BM ,
• Altman DG , et al
• Landes SJ ,
• McBain SA ,
• Schwartz S ,
• Duncan DT , et al
• Ranganathan P
• Proctor E ,
• Silmere H ,
• Raghavan R , et al
• Schwarzer R
• Hägglund M ,
• Atroshi I ,
• Wagner P , et al
• Steffen K ,
• Romiti M , et al
• Sugimoto D ,
• Barber Foss KD , et al
• Schneider B , et al
• Meeuwisse WH ,
• Al Attar WSA ,
• Alzahrani H , et al
• Alizadeh MH ,
• Shahrbanian S , et al
• Waldén M , et al
• Akasaka K ,
• Otsudo T , et al
• Leppänen M ,
• Vasankari T , et al
• LaBella CR ,
• Huxford MR ,
• Grissom J , et al
• Loppini M ,
• Berton A , et al
• Parsons JL ,
• Carswell J ,
• Nwoba IM , et al
• Rössler R ,
• Bizzini M , et al
• Slauterbeck JR ,
• Choquette R ,
• Tourville TW , et al
• Olsen OE , et al
• Mattacola CG ,
• Bush HM , et al
• Hancock MV ,
• Stokes KA , et al
• RÄisÄnen AM , et al
• Barboza SD ,
• Emery C , et al
• Hellquist E ,
• Ahlqvist K , et al
• Tomsovsky L ,
• Whatman C , et al
• Verhagen E ,
• Gouttebarge V , et al
• Celebrini RG ,
• Miller WC , et al
• Franchina M ,
• Tercier S , et al
• Register-Mihalik J ,
• Choquette R , et al
• Lindblom H ,
• Carlfjord S ,
• Ljunggren G ,
• Perera NKP ,
• Merrett CK ,
• Linnéll J , et al
• Myklebust G , et al
• O’Brien J ,
• Santner E ,
• Befus K , et al
• Räisänen A ,
• Black AM , et al
• Sonesson S ,
• Lindblom H , et al
• Nilstad A ,
• Thein-Nissenbaum J ,
• Ageberg E ,
• Lucander K , et al
• Brodin EM ,
• Watkins R ,
• Cornelissen MH ,
• Baan A , et al
• Donaldson A ,
• Callaghan A ,
• Taylor JR ,
• Novak MA , et al
• MacFarlane AJ ,
• Weiss-Laxer NS , et al
• McGuine TA ,
• Pennuto A , et al
• Munoz-Plaza C ,
• Davis A , et al
• Norcross MF ,
• Johnson ST ,
• Bovbjerg VE , et al
• Jimenez P ,
• Oliver JL , et al
• Denegar CR , et al
• Lambert M , et al
• Withall AL , et al
• Minnig MC ,
• Hawkinson L ,
• Root HJ , et al
• Macpherson A , et al
• Fenelon D , et al
• Bruinsma A , et al
• Verhagen EALM , et al
• Brown JC , et al
• Bogardus RL ,
• Martin RJ ,
• Richman AR , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2

X @carlavdb_, @amraisanen, @KatiPasanen, @Kat_Schneider7, @CarolynAEmery, @owoeye_oba

Contributors DL, CE and OBAO contributed to development of study proposal and design. DL, CvdB, AMR, IJS, KV, JK, AH, CE and OBAO conducted search, study selection and screening, data extraction and synthesis and quality assessment. DL led the writing of the manuscript and was the guarantor for the project. All authors contributed to drafting and revising the final manuscript. All authors approved the submitted version of the manuscript.

Funding This study was funded by Canadian Institutes for Health Research Foundation Grant Program (PI CAE).

Competing interests OBAO is a Deputy Editor for the British Journal of Sports Medicine. CE, KJS and KP are Associate Editors for the British Journal of Sports Medicine.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: