limitations research articles

How to Write Limitations of the Study (with examples)

This blog emphasizes the importance of recognizing and effectively writing about limitations in research. It discusses the types of limitations, their significance, and provides guidelines for writing about them, highlighting their role in advancing scholarly research.

Updated on August 24, 2023

No matter how well thought out, every research endeavor encounters challenges. There is simply no way to predict all possible variances throughout the process.

These uncharted boundaries and abrupt constraints are known as limitations in research . Identifying and acknowledging limitations is crucial for conducting rigorous studies. Limitations provide context and shed light on gaps in the prevailing inquiry and literature.

This article explores the importance of recognizing limitations and discusses how to write them effectively. By interpreting limitations in research and considering prevalent examples, we aim to reframe the perception from shameful mistakes to respectable revelations.

What are limitations in research?

In the clearest terms, research limitations are the practical or theoretical shortcomings of a study that are often outside of the researcher’s control . While these weaknesses limit the generalizability of a study’s conclusions, they also present a foundation for future research.

Sometimes limitations arise from tangible circumstances like time and funding constraints, or equipment and participant availability. Other times the rationale is more obscure and buried within the research design. Common types of limitations and their ramifications include:

• Theoretical: limits the scope, depth, or applicability of a study.
• Methodological: limits the quality, quantity, or diversity of the data.
• Empirical: limits the representativeness, validity, or reliability of the data.
• Analytical: limits the accuracy, completeness, or significance of the findings.
• Ethical: limits the access, consent, or confidentiality of the data.

Regardless of how, when, or why they arise, limitations are a natural part of the research process and should never be ignored . Like all other aspects, they are vital in their own purpose.

Why is identifying limitations important?

Whether to seek acceptance or avoid struggle, humans often instinctively hide flaws and mistakes. Merging this thought process into research by attempting to hide limitations, however, is a bad idea. It has the potential to negate the validity of outcomes and damage the reputation of scholars.

By identifying and addressing limitations throughout a project, researchers strengthen their arguments and curtail the chance of peer censure based on overlooked mistakes. Pointing out these flaws shows an understanding of variable limits and a scrupulous research process.

Showing awareness of and taking responsibility for a project’s boundaries and challenges validates the integrity and transparency of a researcher. It further demonstrates the researchers understand the applicable literature and have thoroughly evaluated their chosen research methods.

Presenting limitations also benefits the readers by providing context for research findings. It guides them to interpret the project’s conclusions only within the scope of very specific conditions. By allowing for an appropriate generalization of the findings that is accurately confined by research boundaries and is not too broad, limitations boost a study’s credibility .

Limitations are true assets to the research process. They highlight opportunities for future research. When researchers identify the limitations of their particular approach to a study question, they enable precise transferability and improve chances for reproducibility. 

Simply stating a project’s limitations is not adequate for spurring further research, though. To spark the interest of other researchers, these acknowledgements must come with thorough explanations regarding how the limitations affected the current study and how they can potentially be overcome with amended methods.

How to write limitations

Typically, the information about a study’s limitations is situated either at the beginning of the discussion section to provide context for readers or at the conclusion of the discussion section to acknowledge the need for further research. However, it varies depending upon the target journal or publication guidelines. 

Don’t hide your limitations

It is also important to not bury a limitation in the body of the paper unless it has a unique connection to a topic in that section. If so, it needs to be reiterated with the other limitations or at the conclusion of the discussion section. Wherever it is included in the manuscript, ensure that the limitations section is prominently positioned and clearly introduced.

While maintaining transparency by disclosing limitations means taking a comprehensive approach, it is not necessary to discuss everything that could have potentially gone wrong during the research study. If there is no commitment to investigation in the introduction, it is unnecessary to consider the issue a limitation to the research. Wholly consider the term ‘limitations’ and ask, “Did it significantly change or limit the possible outcomes?” Then, qualify the occurrence as either a limitation to include in the current manuscript or as an idea to note for other projects. 

Writing limitations

Once the limitations are concretely identified and it is decided where they will be included in the paper, researchers are ready for the writing task. Including only what is pertinent, keeping explanations detailed but concise, and employing the following guidelines is key for crafting valuable limitations:

1) Identify and describe the limitations : Clearly introduce the limitation by classifying its form and specifying its origin. For example:

• An unintentional bias encountered during data collection
• An intentional use of unplanned post-hoc data analysis

2) Explain the implications : Describe how the limitation potentially influences the study’s findings and how the validity and generalizability are subsequently impacted. Provide examples and evidence to support claims of the limitations’ effects without making excuses or exaggerating their impact. Overall, be transparent and objective in presenting the limitations, without undermining the significance of the research. 

3) Provide alternative approaches for future studies : Offer specific suggestions for potential improvements or avenues for further investigation. Demonstrate a proactive approach by encouraging future research that addresses the identified gaps and, therefore, expands the knowledge base.

Whether presenting limitations as an individual section within the manuscript or as a subtopic in the discussion area, authors should use clear headings and straightforward language to facilitate readability. There is no need to complicate limitations with jargon, computations, or complex datasets.

Examples of common limitations

Limitations are generally grouped into two categories , methodology and research process .

Methodology limitations

Methodology may include limitations due to:

• Sample size
• Lack of available or reliable data
• Lack of prior research studies on the topic
• Measure used to collect the data
• Self-reported data

The researcher is addressing how the large sample size requires a reassessment of the measures used to collect and analyze the data.

Research process limitations

Limitations during the research process may arise from:

• Access to information
• Longitudinal effects
• Cultural and other biases
• Language fluency
• Time constraints

The author is pointing out that the model’s estimates are based on potentially biased observational studies.

Final thoughts

Successfully proving theories and touting great achievements are only two very narrow goals of scholarly research. The true passion and greatest efforts of researchers comes more in the form of confronting assumptions and exploring the obscure.

In many ways, recognizing and sharing the limitations of a research study both allows for and encourages this type of discovery that continuously pushes research forward. By using limitations to provide a transparent account of the project's boundaries and to contextualize the findings, researchers pave the way for even more robust and impactful research in the future.

Charla Viera, MS

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

• Print this article

Limited by our limitations

• Nikki L. Bibler Zaidi
• Nikki L. Bibler Zaidi , Medical School, University of Michigan, United States

Study limitations represent weaknesses within a research design that may influence outcomes and conclusions of the research. Researchers have an obligation to the academic community to present complete and honest limitations of a presented study. Too often, authors use generic descriptions to describe study limitations. Including redundant or irrelevant limitations is an ineffective use of the already limited word count. A meaningful presentation of study limitations should describe the potential limitation, explain the implication of the limitation, provide possible alternative approaches, and describe steps taken to mitigate the limitation. This includes placing research findings within their proper context to ensure readers do not overemphasize or minimize findings. A more complete presentation will enrich the readers’ understanding of the study’s limitations and support future investigation.

• Page/Article: 261-264
• DOI: 10.1007/S40037-019-00530-X
• Peer Reviewed

Join thousands of product people at Insight Out Conf on April 11. Register free.

Insights hub solutions

Analyze data

Uncover deep customer insights with fast, powerful features, store insights, curate and manage insights in one searchable platform, scale research, unlock the potential of customer insights at enterprise scale.

Featured reads

Inspiration

Three things to look forward to at Insight Out

Tips and tricks

Make magic with your customer data in Dovetail

Four ways Dovetail helps Product Managers master continuous product discovery

Events and videos

© Dovetail Research Pty. Ltd.

How to present limitations in research

Last updated

30 January 2024

Reviewed by

Limitations don’t invalidate or diminish your results, but it’s best to acknowledge them. This will enable you to address any questions your study failed to answer because of them.

In this guide, learn how to recognize, present, and overcome limitations in research.

• What is a research limitation?

Research limitations are weaknesses in your research design or execution that may have impacted outcomes and conclusions. Uncovering limitations doesn’t necessarily indicate poor research design—it just means you encountered challenges you couldn’t have anticipated that limited your research efforts.

Does basic research have limitations?

Basic research aims to provide more information about your research topic. It requires the same standard research methodology and data collection efforts as any other research type, and it can also have limitations.

• Common research limitations

Researchers encounter common limitations when embarking on a study. Limitations can occur in relation to the methods you apply or the research process you design. They could also be connected to you as the researcher.

Methodology limitations

Not having access to data or reliable information can impact the methods used to facilitate your research. A lack of data or reliability may limit the parameters of your study area and the extent of your exploration.

Your sample size may also be affected because you won’t have any direction on how big or small it should be and who or what you should include. Having too few participants won’t adequately represent the population or groups of people needed to draw meaningful conclusions.

Research process limitations

The study’s design can impose constraints on the process. For example, as you’re conducting the research, issues may arise that don’t conform to the data collection methodology you developed. You may not realize until well into the process that you should have incorporated more specific questions or comprehensive experiments to generate the data you need to have confidence in your results.

Constraints on resources can also have an impact. Being limited on participants or participation incentives may limit your sample sizes. Insufficient tools, equipment, and materials to conduct a thorough study may also be a factor.

Common researcher limitations

Here are some of the common researcher limitations you may encounter:

Time: some research areas require multi-year longitudinal approaches, but you might not be able to dedicate that much time. Imagine you want to measure how much memory a person loses as they age. This may involve conducting multiple tests on a sample of participants over 20–30 years, which may be impossible.

Bias: researchers can consciously or unconsciously apply bias to their research. Biases can contribute to relying on research sources and methodologies that will only support your beliefs about the research you’re embarking on. You might also omit relevant issues or participants from the scope of your study because of your biases.

Limited access to data : you may need to pay to access specific databases or journals that would be helpful to your research process. You might also need to gain information from certain people or organizations but have limited access to them. These cases require readjusting your process and explaining why your findings are still reliable.

• Why is it important to identify limitations?

Identifying limitations adds credibility to research and provides a deeper understanding of how you arrived at your conclusions.

Constraints may have prevented you from collecting specific data or information you hoped would prove or disprove your hypothesis or provide a more comprehensive understanding of your research topic.

However, identifying the limitations contributing to your conclusions can inspire further research efforts that help gather more substantial information and data.

• Where to put limitations in a research paper

A research paper is broken up into different sections that appear in the following order:

Introduction

Methodology

The discussion portion of your paper explores your findings and puts them in the context of the overall research. Either place research limitations at the beginning of the discussion section before the analysis of your findings or at the end of the section to indicate that further research needs to be pursued.

What not to include in the limitations section

Evidence that doesn’t support your hypothesis is not a limitation, so you shouldn’t include it in the limitation section. Don’t just list limitations and their degree of severity without further explanation.

• How to present limitations

You’ll want to present the limitations of your study in a way that doesn’t diminish the validity of your research and leave the reader wondering if your results and conclusions have been compromised.

Include only the limitations that directly relate to and impact how you addressed your research questions. Following a specific format enables the reader to develop an understanding of the weaknesses within the context of your findings without doubting the quality and integrity of your research.

Identify the limitations specific to your study

You don’t have to identify every possible limitation that might have occurred during your research process. Only identify those that may have influenced the quality of your findings and your ability to answer your research question.

Explain study limitations in detail

This explanation should be the most significant portion of your limitation section.

Link each limitation with an interpretation and appraisal of their impact on the study. You’ll have to evaluate and explain whether the error, method, or validity issues influenced the study’s outcome and how.

Propose a direction for future studies and present alternatives

In this section, suggest how researchers can avoid the pitfalls you experienced during your research process.

If an issue with methodology was a limitation, propose alternate methods that may help with a smoother and more conclusive research project. Discuss the pros and cons of your alternate recommendation.

Describe steps taken to minimize each limitation

You probably took steps to try to address or mitigate limitations when you noticed them throughout the course of your research project. Describe these steps in the limitation section.

• Limitation example

“Approaches like stem cell transplantation and vaccination in AD [Alzheimer’s disease] work on a cellular or molecular level in the laboratory. However, translation into clinical settings will remain a challenge for the next decade.”

The authors are saying that even though these methods showed promise in helping people with memory loss when conducted in the lab (in other words, using animal studies), more studies are needed. These may be controlled clinical trials, for example. 

However, the short life span of stem cells outside the lab and the vaccination’s severe inflammatory side effects are limitations. Researchers won’t be able to conduct clinical trials until these issues are overcome.

• How to overcome limitations in research

You’ve already started on the road to overcoming limitations in research by acknowledging that they exist. However, you need to ensure readers don’t mistake weaknesses for errors within your research design.

To do this, you’ll need to justify and explain your rationale for the methods, research design, and analysis tools you chose and how you noticed they may have presented limitations.

Your readers need to know that even when limitations presented themselves, you followed best practices and the ethical standards of your field. You didn’t violate any rules and regulations during your research process.

You’ll also want to reinforce the validity of your conclusions and results with multiple sources, methods, and perspectives. This prevents readers from assuming your findings were derived from a single or biased source.

• Learning and improving starts with limitations in research

Dealing with limitations with transparency and integrity helps identify areas for future improvements and developments. It’s a learning process, providing valuable insights into how you can improve methodologies, expand sample sizes, or explore alternate approaches to further support the validity of your findings.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 17 February 2024

Last updated: 19 November 2023

Last updated: 5 March 2024

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

• Open access
• Published: 23 February 2012

Discussing study limitations in reports of biomedical studies- the need for more transparency

• Milo A Puhan 1 ,
• Elie A Akl 2 ,
• Dianne Bryant 3 ,
• Feng Xie 4 ,
• Giovanni Apolone 5 &
• Gerben ter Riet 6  

Health and Quality of Life Outcomes volume  10 , Article number:  23 ( 2012 ) Cite this article

29k Accesses

29 Citations

12 Altmetric

Metrics details

Unbiased and frank discussion of study limitations by authors represents a crucial part of the scientific discourse and progress. In today's culture of publishing many authors or scientific teams probably balance 'utter honesty' when discussing limitations of their research with the risk of being unable to publish their work. Currently, too few papers in the medical literature frankly discuss how limitations could have affected the study findings and interpretations. The goals of this commentary are to review how limitations are currently acknowledged in the medical literature, to discuss the implications of limitations in biomedical studies, and to make suggestions as to how to openly discuss limitations for scientists submitting their papers to journals. This commentary was developed through discussion and logical arguments by the authors who are doing research in the area of hedging (use of language to express uncertainty) and who have extensive experience as authors and editors of biomedical papers. We strongly encourage authors to report on all potentially important limitations that may have affected the quality and interpretation of the evidence being presented. This will not only benefit science but also offers incentives for authors: If not all important limitations are acknowledged readers and reviewers of scientific articles may perceive that the authors were unaware of them. Authors should take advantage of their content knowledge and familiarity with the study to prevent misinterpretations of the limitations by reviewers and readers. Articles discussing limitations help shape the future research agenda and are likely to be cited because they have informed the design and conduct of future studies. Instead of perceiving acknowledgment of limitations negatively, authors, reviewers and editors should recognize the potential of a frank and unbiased discussion of study limitations that should not jeopardize acceptance of manuscripts.

Introduction

The physicist Richard Feynman argued, during his commencement address at the California Institute of Technology in 1974, that utter honesty must be a cornerstone of scientific integrity. He cautioned researchers from fooling themselves by saying: "We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work."[ 1 ]

We think that, in today's culture of publishing biomedical studies, many authors may not want to discuss limitations of their studies because they perceive a transparency threshold beyond which the probability of manuscript acceptance goes down (perhaps even to zero) [ 2 ]. The goals of this commentary are to briefly review how limitations are currently acknowledged in the biomedical literature, to discuss implications of limitations in biomedical studies, and to make suggestions as to how to openly discuss limitations for scientists who submitting their papers to biomedical journals. This commentary was initiated by two of the authors (MP and GtR), who are doing research in the area of hedging (use of language to express uncertainty), and proposed to the editors of Health and Quality of Life Outcomes . All editors supported the idea of writing a commentary on the importance of discussing limitations transparently and four editors (EAA, DB, FX, GA) joined the writing group. This commentary was developed through discussion and logical arguments by the authors who have extensive experience as authors and editors of biomedical papers themselves.

Recognition, acknowledgment and discussion of all potentially important limitations by authors, if presented in an unbiased way, represent a crucial part of the scientific discourse and progress. The advantages of openly discussing limitations are probably long-term and benefit the scientific community and other users of the evidence: A candid discussion of limitations helps readers to correctly interpret the particular study. Conflicting results across studies may be explained by the patterns in limitations. Moreover, frank discussion of limitations informs future studies, which are likely to be of higher quality if they address the limitations of earlier studies. However, while encouraging others to openly discuss limitations of their studies is easy, discussing the limitations of one's own study is more challenging. Researchers usually have their opinion about how to design and execute studies or how to interpret the results and may not agree that some aspects of a study represent, in the view of others, a limitation. Risks of acknowledging limitations and having an open scientific discourse may include, at least in today's culture, eliciting negative comments by peer reviewers, non-acceptance by journals and a potentially negative image as a researcher.

Discussion of limitations in the medical literature

There is some evidence that limitations are not thoroughly discussed in the medical literature. A study using automated key word searching found that only 17% out of 400 papers published in leading medical journals used at least one word referring to limitations [ 3 ]. Not a single article discussed how a limitation could affect the conclusion. In a more detailed assessment of the medical literature, in which two independent reviewers assessed the abstract and discussion sections of 300 medical research papers, published in first and second tier general medical and specialty journals, 73% of all papers were found to acknowledge a median of 3 limitations [ 4 ]. This higher proportion (compared to the first study) is likely due to a more thorough assessment (i.e., by reviewers rather than an automated search) but could also be related to a slightly different selection of papers. The detailed assessment of these 300 papers revealed that 62% of all limitations referred to aspects of internal validity, which could systematically distort the results. Measurement errors, failure to measure important variables and potential confounding were among those acknowledged most frequently. The remaining limitations referred to aspects of applicability of the results to clinical practice (external validity). Differences between the study population and real-world populations were mentioned most frequently as barriers for applying the results in practice. Few authors discussed how the limitations could have affected the interpretation of study findings.

What is currently unclear is whether authors do or do not address those limitations that are most likely to affect internal validity and applicability of results in real practice. It may well be possible that authors discuss limitations because it is required by journal policies and worry that too open discussion jeopardizes the chances of acceptance. Also, more research is needed to see how the acknowledgment of potentially important limitations fits with the claims made in an article, for example about the effectiveness of a medical intervention or about the measurement properties of a patient-reported outcome.

It is time to discuss limitations not in isolation but in the context of the entire article and as part of a rhetorical-epistemic phenomenon that linguists call "hedging." Hedging refers to "the means by which writers can present a proposition as an opinion rather than a fact" [ 5 ]. By using hedging authors can express the extent of uncertainty about the importance and validity of their study but also prevent readers from making false accusations for strong or definitive statements. Of note, hedging has both positive and negative connotations since it can be used to set an appropriate tone but also to express an opinion that may not be fully supported by the facts.

Discussing implications of limitations prevents misunderstandings and supports interpretation of data

It requires a great deal of judgment to estimate the potential impact of limitations on internal or external validity of a study. Sometimes, the direction of bias may be towards an over- or underestimation of effects. For example, if there is systematic measurement error that equally affects different study groups (so called non-differential measurement error, for example if the exposure is measured with a sensitivity of 80% and a specificity of 90%) the results are usually biased towards an underestimation of the effect [ 6 ]. Or, if a confounder is positively associated with the outcome and more prevalent in study participants exposed to the risk factor of interest, an overestimation of the effect can be expected. Some biases, for example selection bias and some forms of measurement error can, affect the results in a direction that is difficult to predict [ 6 ]. Sometimes, the impact of biases on internal validity may be so small that its description may not be warranted.

Very often the authors of an article are in the best position to judge the direction of a potential bias because they executed the study and have experienced first-hand limitations of their study. In addition, they often have the needed content knowledge that would inform the direction and potential extent of bias. Thus authors should acknowledge recognized limitations and discuss their likely implications on the interpretations of the findings; by doing so, they reduce the probability that readers will misjudge the validity and impact of their study. Of course, it is important that the authors also include the reasoning behind their judgment of the magnitude and direction of the potential bias to enable readers to form their own opinion on the impact of limitations.

For some limitations, however, the impact can better be judged in a meta-epidemiological context, that is, when all studies addressing the same research questions are analyzed together. Some journals ask authors to discuss their results in reference to an existing systematic review [ 7 ]. Thereby, not only heterogeneity of results across studies can be detected but it may be possible to estimate how much a limitation may affect the results. For example, a randomized trial may use a generic health-related quality of life instrument to evaluate the effectiveness of a treatment. The trial may show no effect and have high internal validity. However, other trials evaluating the same treatment may have used a disease-specific instrument and shown an effect that exceeded the minimal important difference. Or, studies may have shown that disease-specific instruments discriminate better between disease severity or change over time than generic instruments [ 8 , 9 ]. The limitation of the first trial that used a much less responsive generic instrument only becomes much clearer in a meta-epidemiological context. Another important purpose of systematic reviews is to identify limitations of existing studies and to help investigators to avoid them in the future. It is beyond the scope of this commentary to discuss different types of biases and their implications for the quality of evidence but we refer readers to the extensive literature on biases and to some approaches that are currently used to judge the implications of limitations on the strength of evidence [ 6 , 10 – 12 ].

An open discussion of limitations should not jeopardize paper acceptance by journals

We would like to strongly encourage authors submitting their articles to biomedical journals to openly discuss all potentially relevant limitations of their study. Specifically, we suggest including text in the abstract and discussion section (Table 1 ):

At the end of the results section add one sentence highlighting the one or two main limitations of the study. The conclusion section should reflect the seriousness of the limitations as perceived by the authors and their potential impact on the results and interpretation of the study.

Discussion section

Report on all limitations that may have affected the quality of the evidence being presented, including aspects of study design and implementation. Readers depend on a candid communication by the authors and may get the impression that the investigators were naive if they are not reported. If space is limited an online appendix could be considered that describes the limitations as well as their potential implications in more details.

Give the authors' view on how the limitations impact on the quality of the evidence and discuss the direction and magnitude of bias. For example, a recent study reporting on the association of quality of life of elderly people with nursing home placement and death discussed the potential mechanism of a selection bias by economic status. The authors concluded that a selection bias based on economic status was unlikely because access to health care, and thus selection into the study, did not depend on economic status [ 13 ]. As explained above, few authors currently discuss how limitations could have affected the strength of the conclusions that may be drawn. However, the authors should take advantage of their content knowledge and familiarity with the study and the meta-epidemiological context to prevent misinterpretations of the limitations by reviewers and readers.

Do not restrict the discussion of limitations to aspects of internal validity. For readers, it is important to learn about potential barriers for applying the evidence, generated in scientific studies, to practice. Discuss where the limits of applicability of the results may lie. This requires a discussion of the setting in which the study took place, how and why the results may differ in another setting (potential effect modification) and what barriers may exist to adopt new interventions or diagnostic procedures in a setting that is different from the research setting [ 14 ].

Discuss the strengths of the study that may counterbalance or outweigh (some of) the limitations. Be explicit about the strengths, in particular how the study was implemented, and do not limit the discussion of strengths to general statements about study design.

Provide suggestions for future research specifically overcoming the limitations of the current study. One may also consider describing how one's own study could be repeated and conducted differently to avoid some of the limitations. Articles acknowledging and putting into context all potentially relevant limitations could help shape the research agenda and may be more likely to be cited because they inform the design and conduct of future studies.

We acknowledge that, even if limitations are openly discussed, some articles will be rejected by journals because the limitations affect an article's validity, level of interest to the reader and comprehensibility too much as assessed by peer reviewers. But we believe that journal editors should consider the thoroughness with which limitations are discussed in their editorial decisions on acceptance. In fact, editors should consider it a shortcoming of the submission if a candid discussion is lacking. To end with Feynman's words, "[...] if you are doing an experiment, you should report everything that you think might make it invalid - not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked - to make sure the other fellow can tell they have been eliminated."[ 1 ]

Feynman RP: Cargo Cult Science. Eng Sci 1974, 37(7):10–13.

Google Scholar  

Montori VM, Jaeschke R, Schunemann HJ, Bhandari M, Brozek JL, Devereaux PJ, Guyatt GH: Users' guide to detecting misleading claims in clinical research reports. BMJ 2004, 329(7474):1093–1096. 10.1136/bmj.329.7474.1093

Article   PubMed Central   PubMed   Google Scholar  

Ioannidis JP: Limitations are not properly acknowledged in the scientific literature. J Clin Epidemiol 2007, 60(4):324–329. 10.1016/j.jclinepi.2006.09.011

Article   PubMed   Google Scholar  

Puhan MA, Heller N, Joleska I, Siebeling L, Muggensturm P, Umbehr M, Goodman S, ter Riet G: Acknowledging Limitations in Biomedical Studies: The ALIBI Study. In The Sixth International Congress on Peer Review and Biomedical Publication . Vancouver, Canada: JAMA and BMJ; 2009.

Hyland K: Hedging in scientific research articles . Amsterdam and Philadelphia: John Benjamins Publication Company; 1998.

Book   Google Scholar  

Rothman KJ, Greenland S, Lash TL: Modern Epidemiology . 3rd edition. Lippincott Williams & Wilkins; 2008.

Clark S, Horton R: Putting research into context--revisited. Lancet 2010, 376(9734):10–11. 10.1016/S0140-6736(10)61001-X

Puhan MA, Guyatt GH, Goldstein R, Mador J, McKim D, Stahl E, Griffith L, Schunemann HJ: Relative responsiveness of the Chronic Respiratory Questionnaire, St. Georges Respiratory Questionnaire and four other health-related quality of life instruments for patients with chronic lung disease. Respir Med 2007, 101(2):308–316. 10.1016/j.rmed.2006.04.023

Teckle P, Peacock S, McTaggart-Cowan H, van der Hoek K, Chia S, Melosky B, Gelmon K: The ability of cancer-specific and generic preference-based instruments to discriminate across clinical and self-reported measures of cancer severities. Health Qual Life Outcomes 2011, 9: 106. 10.1186/1477-7525-9-106

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ: What is "quality of evidence" and why is it important to clinicians? BMJ 2008, 336(7651):995–998. 10.1136/bmj.39490.551019.BE

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schunemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336(7650):924–926. 10.1136/bmj.39489.470347.AD

Haynes RB, Sackett D, Tugwell P, Guyatt GH: Clinical Epidemiology: How to Do Clinical Practice Research. 3rd edition. Lippincott Williams & Wilkins; 2005.

Bilotta C, Bowling A, Nicolini P, Case A, Pina G, Rossi SV, Vergani C: Older People's Quality of Life (OPQOL) scores and adverse health outcomes at a one-year follow-up. A prospective cohort study on older outpatients living in the community in Italy. Health Qual Life Outcomes 2011, 9: 72. 10.1186/1477-7525-9-72

Bausewein C, Simon ST, Benalia H, Downing J, Mwangi-Powell FN, Daveson BA, Harding R, Higginson IJ: Implementing patient reported outcome measures (PROMs) in palliative care--users' cry for help. Health Qual Life Outcomes 2011, 9: 27. 10.1186/1477-7525-9-27

Download references

Author information

Authors and affiliations.

Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, 615 North Wolfe Street, Mail room E6153, Baltimore, MD, 21205, USA

Milo A Puhan

Department of Medicine, State University of New York at Buffalo, Buffalo, NY, USA

Faculty of Health Sciences, University of Western Ontario, London, Canada

Dianne Bryant

Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada

Scientific Directorate, Arcispedale Santa Maria Nuova, IRCCS, Reggio Emilia, Italy

Giovanni Apolone

Department of General Practice, Academic Medical Center University of Amsterdam, Amsterdam, The Netherlands

Gerben ter Riet

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Milo A Puhan .

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Puhan, M.A., Akl, E.A., Bryant, D. et al. Discussing study limitations in reports of biomedical studies- the need for more transparency. Health Qual Life Outcomes 10 , 23 (2012). https://doi.org/10.1186/1477-7525-10-23

Download citation

Received : 15 March 2011

Accepted : 23 February 2012

Published : 23 February 2012

DOI : https://doi.org/10.1186/1477-7525-10-23

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Medical Literature
• Content Knowledge
• Internal Validity
• Minimal Important Difference
• Scientific Discourse

Health and Quality of Life Outcomes

ISSN: 1477-7525

• Submission enquiries: [email protected]

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Limitations of the Study
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. Study limitations are the constraints placed on the ability to generalize from the results, to further describe applications to practice, and/or related to the utility of findings that are the result of the ways in which you initially chose to design the study or the method used to establish internal and external validity or the result of unanticipated challenges that emerged during the study.

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Theofanidis, Dimitrios and Antigoni Fountouki. "Limitations and Delimitations in the Research Process." Perioperative Nursing 7 (September-December 2018): 155-163. .

Importance of...

Always acknowledge a study's limitations. It is far better that you identify and acknowledge your study’s limitations than to have them pointed out by your professor and have your grade lowered because you appeared to have ignored them or didn't realize they existed.

Keep in mind that acknowledgment of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgment of a study's limitations also provides you with opportunities to demonstrate that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the results and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in the introduction of your paper.

Here are examples of limitations related to methodology and the research process you may need to describe and discuss how they possibly impacted your results. Note that descriptions of limitations should be stated in the past tense because they were discovered after you completed your research.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred. Note that sample size is generally less relevant in qualitative research if explained in the context of the research problem.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but provide cogent reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe a need for future research based on designing a different method for gathering data.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, though, consult with a librarian! In cases when a librarian has confirmed that there is little or no prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design ]. Note again that discovering a limitation can serve as an important opportunity to identify new gaps in the literature and to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need for future researchers to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to the accuracy of what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data can contain several potential sources of bias that you should be alert to and note as limitations. These biases become apparent if they are incongruent with data from other sources. These are: (1) selective memory [remembering or not remembering experiences or events that occurred at some point in the past]; (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency, but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, data, or documents and, for whatever reason, access is denied or limited in some way, the reasons for this needs to be described. Also, include an explanation why being denied or limited access did not prevent you from following through on your study.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single topic, the time available to investigate a research problem and to measure change or stability over time is constrained by the due date of your assignment. Be sure to choose a research problem that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure whether you can complete your research within the confines of the assignment's due date, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, event, or thing is viewed or shown in a consistently inaccurate way. Bias is usually negative, though one can have a positive bias as well, especially if that bias reflects your reliance on research that only support your hypothesis. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places, how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. NOTE :   If you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating that bias. For example, if a previous study only used boys to examine how music education supports effective math skills, describe how your research expands the study to include girls.
• Fluency in a language -- if your research focuses , for example, on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic or to speak with these students in their primary language. This deficiency should be acknowledged.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods. Powerpoint Presentation. Regent University of Science and Technology; ter Riet, Gerben et al. “All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies.” PLOS One 8 (November 2013): 1-6.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as an exploratory study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in a new study.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to revise your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to acquire or gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't interview a group of people that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in a future study. A underlying goal of scholarly research is not only to show what works, but to demonstrate what doesn't work or what needs further clarification.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. "Limitations are not Properly Acknowledged in the Scientific Literature." Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed. January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings!

After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitations of your study. Inflating the importance of your study's findings could be perceived by your readers as an attempt hide its flaws or encourage a biased interpretation of the results. A small measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated. Or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may very well be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Lewis, George H. and Jonathan F. Lewis. “The Dog in the Night-Time: Negative Evidence in Social Research.” The British Journal of Sociology 31 (December 1980): 544-558.

Yet Another Writing Tip

Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgment about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Boddy, Clive Roland. "Sample Size for Qualitative Research." Qualitative Market Research: An International Journal 19 (2016): 426-432; Huberman, A. Michael and Matthew B. Miles. "Data Management and Analysis Methods." In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444; Blaikie, Norman. "Confounding Issues Related to Determining Sample Size in Qualitative Research." International Journal of Social Research Methodology 21 (2018): 635-641; Oppong, Steward Harrison. "The Problem of Sampling in qualitative Research." Asian Journal of Management Sciences and Education 2 (2013): 202-210.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: Mar 26, 2024 10:40 AM
• URL: https://libguides.usc.edu/writingguide

• Open access
• Published: 16 September 2019

Impact of peer review on discussion of study limitations and strength of claims in randomized trial reports: a before and after study

• Kerem Keserlioglu 1 ,
• Halil Kilicoglu 2 &
• Gerben ter Riet   ORCID: orcid.org/0000-0002-2231-7637 3 , 4  

Research Integrity and Peer Review volume  4 , Article number:  19 ( 2019 ) Cite this article

8437 Accesses

2 Citations

84 Altmetric

Metrics details

In their research reports, scientists are expected to discuss limitations that their studies have. Previous research showed that often, such discussion is absent. Also, many journals emphasize the importance of avoiding overstatement of claims. We wanted to see to what extent editorial handling and peer review affects self-acknowledgment of limitations and hedging of claims.

Using software that automatically detects limitation-acknowledging sentences and calculates the level of hedging in sentences, we compared the submitted manuscripts and their ultimate publications of all randomized trials published in 2015 in 27 BioMed Central (BMC) journals and BMJ Open. We used mixed linear and logistic regression models, accounting for clustering of manuscript-publication pairs within journals, to quantify before-after changes in the mean numbers of limitation-acknowledging sentences, in the probability that a manuscript with zero self-acknowledged limitations ended up as a publication with at least one and in hedging scores.

Four hundred forty-six manuscript-publication pairs were analyzed. The median number of manuscripts per journal was 10.5 (interquartile range 6–18). The average number of distinct limitation sentences increased by 1.39 (95% CI 1.09–1.76), from 2.48 in manuscripts to 3.87 in publications. Two hundred two manuscripts (45.3%) did not mention any limitations. Sixty-three (31%, 95% CI 25–38) of these mentioned at least one after peer review. Changes in mean hedging scores were negligible.

Conclusions

Our findings support the idea that editorial handling and peer review lead to more self-acknowledgment of study limitations, but not to changes in linguistic nuance.

Peer Review reports

One of the main functions of the editorial process (peer review and editorial handling) as employed by almost all serious scientific journals is to ensure that the research articles published are accurate, transparent, and complete reports of the research conducted. Spin is a term used to describe reporting practices that distort the interpretation of a study’s results [ 1 ] . Not mentioning (all important) study limitations is one way in which readers can be misguided into believing that, for example, the beneficial effect of an experimental treatment is greater than the trial’s result warrant.

In a survey among scientists, insufficient reporting of study limitations ranked high in a list of detrimental research practices [ 2 ]. In a masked before-after study at the editorial offices of Annals of Internal Medicine, Goodman et al. found that the reporting of study limitations was fairly poor in manuscripts but improved after peer review and editing [ 3 ]. Ter Riet et al. demonstrated that more than a quarter of biomedical research articles do not mention any limitations [ 4 ]. And finally, Horton, in a survey among all authors of ten Lancet papers, found that “Important weaknesses were often admitted on direct questioning, but were not included in the published article” [ 5 ]. Other forms of spin are inappropriate extrapolation of results and inferring causal relationships when the study’s design does not allow for it [ 1 ].

Peer reviewers should spot and suggest changes to overstatements and claims that are too strong and point out non-trivial study weaknesses that are not mentioned. The peer review process may therefore been seen as “a negotiation between authors and journal about the scope of the knowledge claims that will ultimately appear in print” [ 6 ]. Specific words that can be used to add nuance to statements and forestall potential overstatement are so-called “hedges”; these are words like “might,” “could,” “suggest,” “appear,” etc. [ 7 ] Authors of an article are arguably in the best position to point out their study’s weaknesses, but they may feel that naming too many or discussing them too extensively could hurt their chances of publication. In this contribution, we hypothesized that, compared to the subsequent publications, the discussion sections of the submitted manuscripts contain fewer acknowledgments of limitations and are less strongly hedged.

In this study, we considered the discussion sections of randomized clinical trial (RCT) reports published in 27 BioMed Central (BMC) journals and BMJ Open. Using two software tools, we determined the number of sentences dedicated to the acknowledgment of specific study limitations and the use of linguistic hedges, before (manuscripts) and after peer review (publications). The limitation detection tool relies on the structure of the discussion sections and linguistic clues to identify limitation sentences [ 8 ]. In a formal evaluation, its accuracy was found to be 91.5% (95% CI 90.1–92.9). The hedging detection tool uses a lexicon containing 190 weighted hedges. The system computes an overall hedging score based on the number and strength of hedges in a text. Hedge weights range from 1 (low hedging strength, e.g., “largely”) to 5 (high hedging strength, e.g., “may”). The overall hedging score is then divided by the word count of the discussion section (normalization). We also calculated “unweighted” scores, in which all hedges are weighted equally as 1. The software tool yielded 93% accuracy in identifying hedged sentences in a formal evaluation [ 9 ]. The manuscripts were downloaded from the journals’ websites followed by manual pre-processing to restore sentence and paragraph structure. Our software automatically extracted the discussion sections in the publications from PubMed Central.

We also carried out a qualitative analysis of the two publications with the largest increase and decrease of hedging score, respectively. For these two papers, KK compared the before and after discussion sections to see what the actual changes were. The reviewer reports, consisting of the reviewer’s comments and the authors responses, were analyzed.

We performed mixed linear regression analysis, for each manuscript-publication pair, of the mean changes in the number of limitation sentences and normalized hedging scores, with the journal as a random intercept. We repeated these analyses adjusting for the journal’s impact factor (continuous), editorial team size (continuous), and composition of authors in terms of English proficiency (three dummy variables representing four categories). English proficiency was derived from the classification of majority native English-speaking countries by the United Kingdom (UK) government for British citizenship application [ 10 ]. English proficiency was categorized as follows: (i) All authors are residents of an English native country; (ii) the first author is an English native, but at least one co-author is not; (iii) the first author is not an English native, but at least one co-author is; and (iv) none of the authors are English natives. We performed a sensitivity analysis, in which we excluded the manuscript-publication pairs of BMJ Open ( n  = 69) and BMC Medicine ( n  = 14) due to their exceptional number of editorial team members (84 and 182, respectively). Finally, using scatterplots and fractional polynomial functions, we visually explored if the effect on the changes in the number of limitation-acknowledging sentences was affected by the number of limitation-acknowledging sentences in the manuscript controlled for regression to the mean using a median split as suggested by Goodman et al. [ 3 ]. We present the results of the crude and adjusted analyses in Table  2 and those of the sensitivity analyses in Appendix 1 .

We used mixed-effects logistic regression analysis to assess the impact of the abovementioned factors on the likelihood of mentioning at least one limitation in the publication among those that had none in the manuscript. Sensitivity analyses consisted of restricting the data set to the journals with fewer than 20 editorial team members, at least 10 manuscript-publication pairs, and both of those restrictions simultaneously, respectively.

Four hundred forty-six research articles were selected. Table  1 shows a few key journal characteristics. The median number of manuscripts per journal was 10.5 (interquartile range (IQR) 6.5–18.5; range 2–69). Table  2 shows the results. The average number of distinct limitation sentences increased by 1.39, from 2.48 (manuscripts) to 3.87 (publications). Two hundred two manuscripts (45.3%) did not mention any limitations. Sixty-three (31%, 95% CI 25–38) of these mentioned at least one after peer review. Of the 244 manuscripts that mentioned at least one limitation, eight (3%, 95% CI 2–6) mentioned none in the publication. Across the (sensitivity) analyses performed, the probability of mentioning at least one limitation in the publication among those that had none in the manuscript was not consistently associated with any of the three covariables assessed, although higher impact factors tended to be weakly associated with lower probabilities and size of the editorial team weakly with higher probabilities (data not shown). The visual assessment of how the number of changes in the limitation-acknowledging sentences depended on the number of such sentences in the manuscript showed an inverse relation, that is, larger changes were seen in manuscripts with low numbers of limitation-acknowledging sentences (Fig.  1 ).

Changes in the number of limitation-acknowledging sentences between manuscripts and publications as a function of the number such sentences in the manuscript. Left panel: manuscript-publication pairs below the median split. Right panel: manuscript-publication pairs above the median split. The median split was calculated as the average of the number of limitation-acknowledging sentences in the manuscript ( L m ) and in the publication ( L p ): ( L m  +  L p )/2. These averages were ranked and the median (value = 2; interquartile range 0–5) determined. The lines are fitted using fractional polynomials with 95% confidence intervals (Stata 13.1, twoway fpfitci command). Note that, in both panels, the changes tend to increase with decreasing numbers of limitation-acknowledging sentences in the manuscript. In particular, the effect of peer review and editorial handling is large in those manuscripts above the median split (right panel) with zero limitation-acknowledging sentences in the manuscript. The vertical line in the right panel is the line x  = 1. Cluttering of data points was prevented by jittering them. Therefore, data points for x  = 0 are not placed exactly above the tick mark for x  = 0 but somewhat scattered to the left and right. The same holds for all data points and for the vertical placement of the points

The hedging-related differences were all very close to zero. A post hoc analysis inspired by the hypothesis that limitation-acknowledging sentences themselves might affect the average hedging scores confirmed the main analysis.

The largest increase in hedging score was + 1.67 (from 3.33 to 5.00). The weighted hedging scores were 50 across the 15 detected sentences in the manuscript and 145 across the 29 detected sentences in the published paper, respectively. The largest decrease in hedging score was − 2.55 (from 6.85 to 4.30). The weighted hedging score was 192 across 28 sentences in the manuscript and 142 across 33 sentences in the published paper (see Appendix 3 for the textual changes).

In a sample of 446 randomized trial reports published in 28 open access journals, we found a 56% increase in the number of sentences dedicated to study limitations after peer review, although one may argue that in absolute terms, the gain was modest (1.39 additional sentences). Our automated approach showed that 33% of research reports do not contain limitation sentences after peer review. This is comparable with the finding of 27% by Ter Riet et al., which they determined with a manual approach. Goodman et al. found that mentioning study limitations is one of the poorest scoring items before and one of the most improved factors after peer review [ 3 ]. Like Goodman et al., we found evidence that peer review and editorial handling had greater impact on manuscripts with zero and very low numbers of limitation-acknowledging sentences. In Appendix 2 , we highlight the attention to mentioning study limitations in seven major reporting guidelines.

Our findings do not support the hypothesis that the editorial process increases the qualification of claims by using a more nuanced language. The small-scale qualitative analysis of two manuscript-publication pairs indicated that authors are asked to both tone down statements, that is, hedge more strongly, and make statements less speculative, that is hedge less. These phenomena may offset each other resulting in minimal changes in the overall use of hedges (see Appendix 3 for the actual text changes). While the hedging terms and their strength scores were selected based on a careful analysis of the linguistic literature on this topic, it is possible that authors use terms indicating different degrees of certainty (e.g., could vs. may ) somewhat interchangeably. This may explain our finding that the net change in hedging scores was very small.

To better understand the influence of peer review on changes made to manuscripts before publication, it may be interesting to conduct more extensive qualitative analyses of the peer review reports and correspondence available in the files of editorial boards or publishers. Another interesting research avenue may be the comparison of rejected manuscripts to accepted ones, to assess if acknowledgment of limitations and degree of hedging affects acceptance rates. It may be useful to restrict such analyses to sentences in which particular claims on, for example, generalizability are made.

Arguably, our software tools might be utilized by editorial boards (or submitting authors) to flag up particular paragraphs that might deserve more (editorial) attention. The limitation sentence recognizing software could for example be used to alert editors to manuscripts with zero self-acknowledged limitations to see if such omission can be justified. If reference values existed that represented the range of hedging scores across a large body of papers, the hedge-detection software could help inform reviewers (or even authors) that the manuscript has an unusual (weighted) hedging score and let them revisit some the formulations in the paper. We think that currently, no direct conclusions should be drawn from the numbers alone. Human interpretation will remain critical for some time to come, but a signposting role of the software seems currently feasible.

A limitation of our study is that we only included reports or randomized trials that made it to publications. Acknowledgment of limitations among all submissions, including also observational studies, may be different than what we report here. Another limitation is that we only included open peer review journals of more than average editorial team quality. Blind peer review may lead to different results as may the case for journals with lower quality editorial team. Note also that the weight assigned to the hedges is somewhat subjective. However, our results were stable across weighted and unweighted hedges. Finally, one may argue that there is a discrepancy between our interest in overstated claims and what we actually measured, namely, hedging scores in all sentences in the discussion sections. A stricter operationalization of our objective would have required that we detect “claim sentences” first and then measure hedging levels in those sentences only. On the other hand, our approach to focus on discussion sections only is better than analyzing complete papers, because claims are usually made in the discussion sections. A strength of our study is the automated assessment of limitation sentences and hedges, limiting the likelihood of analytical or observational bias. Such automated assessment could also assist journal editors as well as peer reviewers in their review tasks. Our results suggest that reviewers and/or editors demand discussion of study limitations that authors were unaware of or unwilling to discuss. Since good science implies the full disclosure of issues that may (partially) invalidate the findings of a study, this increase in the number of limitation sentences is a positive effect of the peer and editorial review process.

Our findings support the idea that editorial handling and peer review, on average, cause a modest increase in the number of self-acknowledged study limitations and that these effects are larger in a manuscript reporting zero or very few limitations. This finding is important in the debates about the value of peer review and detrimental research practices. Software tools such as the ones used in this study may be employed by authors, reviewers, and editors to flag potentially problematic manuscripts or sections thereof. More research is needed to assess more precisely the effects, if any, of peer review and editorial handling on linguistic nuance of claims.

Availability of data and materials

The datasets used and/or analyzed during the current study as well as the software tools used for the detection of limitation-acknowledging sentences and hedges are available from the corresponding author on reasonable request.

Abbreviations

BioMed Central

Confidence interval

Interquartile range

Randomized clinical trial

Standard deviation

United Kingdom

Chiu K, Grundy Q, Bero L. ‘Spin’ in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9):e2002173.

Article   Google Scholar  

Bouter LM, Tijdink J, Axelsen N, Martinson BC, Ter Riet G. Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Res Integr Peer Rev. 2016;1:17.

Goodman SN, Berlin J, Fletcher SW, Fletcher RH. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Ann Intern Med. 1994;121(1):11–21.

Ter Riet G, Chesley P, Gross AG, Siebeling L, Muggensturm P, Heller N, et al. All that glitters isn’t gold: a survey on acknowledgement of limitations in biomedical studies. PLoS One. 2013;8(11):e73623.

Horton R. The hidden research paper. JAMA. 2002;287(21):2775–8.

Green SM, Callaham ML. Implementation of a journal peer reviewer stratification system based on quality and reliability. Ann Emerg Med. 2011;57(2):149–52 e4.

Hyland K. Hedging in scientific research articles. Philadelphia: John Benjamins Publishing Company; 1998.

Book   Google Scholar  

Kilicoglu H, Rosemblat G, Malicki M, Ter Riet G. Automatic recognition of self-acknowledged limitations in clinical research literature. J Am Med Inform Assoc. 2018;25(7):855–61.

Kilicoglu H, Bergler S. Recognizing speculative language in biomedical research articles: a linguistically motivated perspective. BMC bioinf. 2008;9(Suppl 11):S10.

Prove your knowledge of English for citizenship and settling [Available from: https://www.gov.uk/english-language/exemptions . Accessed June 2018.

Download references

Acknowledgements

We are grateful to the editors of BMC and BMJ Open for their assistance in providing the manuscripts to us, and we are especially grateful to Elizabeth Moylan for her extended help herein.

HK was supported by the intramural research program at the US National Library of Medicine, National Institutes of Health.

Author information

Authors and affiliations.

Department of General Practice, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105AZ, Amsterdam, The Netherlands

Kerem Keserlioglu

Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, Bethesda, MD, USA

Halil Kilicoglu

Department of Cardiology, Amsterdam UMC, University of Amsterdam, Meibergdreef 9, 1105AZ, Amsterdam, The Netherlands

Gerben ter Riet

ACHIEVE Centre for Applied Research, Amsterdam University of Applied Sciences, Tafelbergweg 51, 1105 BD, Amsterdam, The Netherlands

You can also search for this author in PubMed   Google Scholar

Contributions

GtR conceived of the main research idea. HK developed the software tools that detect limitation-acknowledging sentences and hedges and processed the data with these tools. HK and KK extracted the data and checked these for errors. KK collected the data and did qualitative analyses. GtR supervised the project. GtR and KK drafted the current manuscript and analyzed the data. All authors contributed intellectually to the final manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Gerben ter Riet .

Ethics declarations

Ethics approval and consent to participate.

This was a literature-based study and no ethics approval was sought.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Qualitative analysis

The two articles with the largest increase and decrease in their discussion sections’ hedging scores after the editorial process were evaluated. We analyzed the before and after discussion sections as well as the correspondence between reviewers and authors.

The largest increase in a discussion section hedging score was + 1.67 (from 3.33 to 5.00).

There was one clear example of the adding of a hedge.

Manuscript: The present study suggests that there is no evidence of an effect of the completion of a standard informed consent procedure on…

Publication: The present study suggests that there is no strong evidence of an effect of the completion of a standard informed consent procedure on… No other sentences were adjusted. However, three paragraphs were added, mainly consisting of study limitations and nuancing of findings:

Publication: If effects of the type we hypothesised do exist, and we suggest that despite the overall finding, this study can provide some tentative evidence that they do, we may anticipate that they will vary in their magnitude… Reviewer’s comment: “The authors mention in their response that they don’t have access to timing. While I appreciate that, there should be better discussion on this broader point in the manuscript. For example, how would the authors have done things differently? One possibility is to ask a question of the participants who received the intervention that, perhaps indirectly, evaluates whether or not they read the information sheet. In the discussion, the authors have an opportunity to be a little creative in what they suggest.”

Author’s response: “This is very helpful and is now discussed towards the end of the discussion section.”

Publication: Another limitation of the study is that we are unable to determine if the participants assigned to the intervention group actually read the information… Although we could have adopted strategies such as … The absence of any exposure enhancement measures in the present study, also implies some degree of experimental manipulation failure, in that not all randomised participants may have been fully exposed to the possible effects we were seeking measure. This should be borne in mind when interpreting the results of the present study .The largest decrease in a discussion section hedging score was − 2.55 (from 6.85 to 4.30).

Manuscript: Our results contrast with previous animal data by indicating that RIC appears to be an effective adjunct to pPCI in STEMI patients regardless of most cardiovascular risk factors… Reviewer’s comment: “In my opinion there is some over-interpretation in the Discussion. The opening statement that RIC appears to be an effective adjunct to pPCI in STEMI patients is based on a confidence interval with a lower limit of 0. This is of borderline statistical significance.”

Author’s response: “We have revised as recommended and down-graded the opening statement. Additionally, we have specified that the statistical power was limited, and our study should be considered exploratory.”

Publication: Our analysis did not demonstrate significant modification on the efficacy of RIC by cardiovascular risk factors and their medications in patients with STEMI undergoing pPCI. Because the statistical power was limited, our study should be considered exploratory. Reviewer’s comment: “Within each of the subgroups in the discussion, as tests for interaction have not been performed the interpretation is somewhat subjective as to whether there is a difference in RIC effect between the subgroups. In places I feel the interpretation is too strong, and this part of the discussion is too long.”

Manuscript: In our clinical randomised study, we did not find an attenuated effect of RIC in patients with diabetes mellitus or in patients with high plasma glucose or HbA1c levels. Rather, the point estimates tended to support the opposite effect. Antidiabetic drugs may modulate the response to RIC, but because of the limited number of diabetic patients in our study, we were unable to stratify our analysis according to type of antidiabetic treatment.

Publication: The number of patients with diabetes mellitus was limited and our analysis does not allow a conclusion about the modification of the efficacy of RIC in patients with diabetes mellitus.

Manuscript: Our analysis demonstrated that the effect of RIC was preserved among statin users. Our data even may indicate that statin use increased the efficacy of RIC, as suggested by the markedly higher point estimate among statin users, although the confidence intervals were wide. Furthermore, we found that efficacy of RIC was independent of lipid levels at hospital admission.

Publication: Little is known about the effect modification of statin use on RIC. Thus, we are the first to indicate a potential increased effect of RIC in statin users. Whether RIC has a more pronounced effect in statin users deserves further investigation.

Manuscript: It would be instructive to investigate whether RIC and acute beta blocker treatment have additive cardioprotective effects. ACE inhibitors and ARBs have been shown to protect against reperfusion injury in animal models. However, angiotensin II also may be involved in the signaling cascade of ischaemic preconditioning. In a rabbit model, inhibition of the angiotensin II receptor (subtype AT1) with losartan eradicated the effect of local ischaemic conditioning. No studies have investigated the interaction of ACE inhibitor and ARB treatment with RIC, which may act through pathways other than local ischaemic preconditioning. Neither ACE inhibitors nor ARBs seemed to diminish the effect of RIC in our analysis, but additional animal and clinical studies are needed to clarify any potential modifying effect of ACE inhibitor and ARB treatment on RIC.

Publication: This paragraph was removed entirely.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Keserlioglu, K., Kilicoglu, H. & ter Riet, G. Impact of peer review on discussion of study limitations and strength of claims in randomized trial reports: a before and after study. Res Integr Peer Rev 4 , 19 (2019). https://doi.org/10.1186/s41073-019-0078-2

Download citation

Received : 09 February 2019

Accepted : 14 August 2019

Published : 16 September 2019

DOI : https://doi.org/10.1186/s41073-019-0078-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Peer review
• Study limitations
• Before-after study
• Linguistic spin
• Transparency
• Scientific reporting
• Randomized trial

Research Integrity and Peer Review

ISSN: 2058-8615

• Submission enquiries: [email protected]
• General enquiries: [email protected]

English for Writing Research Papers pp 109–118 Cite as

Discussing your limitations

• Adrian Wallwork 3  
• First Online: 21 September 2023

545 Accesses

Part of the book series: English for Academic Research ((EAR))

Chapter 7 highlights the importance to the scientific community of discussing the possible limitations in your research and explains how to present your negative results. Of course, you may have got negative results for other reasons: i) your hypothesis was incorrect and needs to be reformulated, ii) you had a bad experimental design and / or low statistical power. However, this chapter is based on the assumption that both your hypothesis and experimental design were reasonably sound, but still did not produce optimal results.

This is a preview of subscription content, log in via an institution .

Buying options

• Available as EPUB and PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Author information

Authors and affiliations.

Pisa, Italy

Adrian Wallwork

You can also search for this author in PubMed   Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Cite this chapter.

Wallwork, A. (2023). Discussing your limitations. In: English for Writing Research Papers . English for Academic Research. Springer, Cham. https://doi.org/10.1007/978-3-031-31072-0_7

Download citation

DOI : https://doi.org/10.1007/978-3-031-31072-0_7

Published : 21 September 2023

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-31071-3

Online ISBN : 978-3-031-31072-0

eBook Packages : Education Education (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Writing Limitations of Research Study — 4 Reasons Why It Is Important!

It is not unusual for researchers to come across the term limitations of research during their academic paper writing. More often this is interpreted as something terrible. However, when it comes to research study, limitations can help structure the research study better. Therefore, do not underestimate significance of limitations of research study.

Allow us to take you through the context of how to evaluate the limits of your research and conclude an impactful relevance to your results.

Table of Contents

What Are the Limitations of a Research Study?

Every research has its limit and these limitations arise due to restrictions in methodology or research design.  This could impact your entire research or the research paper you wish to publish. Unfortunately, most researchers choose not to discuss their limitations of research fearing it will affect the value of their article in the eyes of readers.

However, it is very important to discuss your study limitations and show it to your target audience (other researchers, journal editors, peer reviewers etc.). It is very important that you provide an explanation of how your research limitations may affect the conclusions and opinions drawn from your research. Moreover, when as an author you state the limitations of research, it shows that you have investigated all the weaknesses of your study and have a deep understanding of the subject. Being honest could impress your readers and mark your study as a sincere effort in research.

Why and Where Should You Include the Research Limitations?

The main goal of your research is to address your research objectives. Conduct experiments, get results and explain those results, and finally justify your research question . It is best to mention the limitations of research in the discussion paragraph of your research article.

At the very beginning of this paragraph, immediately after highlighting the strengths of the research methodology, you should write down your limitations. You can discuss specific points from your research limitations as suggestions for further research in the conclusion of your thesis.

1. Common Limitations of the Researchers

Limitations that are related to the researcher must be mentioned. This will help you gain transparency with your readers. Furthermore, you could provide suggestions on decreasing these limitations in you and your future studies.

2. Limited Access to Information

Your work may involve some institutions and individuals in research, and sometimes you may have problems accessing these institutions. Therefore, you need to redesign and rewrite your work. You must explain your readers the reason for limited access.

3. Limited Time

All researchers are bound by their deadlines when it comes to completing their studies. Sometimes, time constraints can affect your research negatively. However, the best practice is to acknowledge it and mention a requirement for future study to solve the research problem in a better way.

4. Conflict over Biased Views and Personal Issues

Biased views can affect the research. In fact, researchers end up choosing only those results and data that support their main argument, keeping aside the other loose ends of the research.

Types of Limitations of Research

Before beginning your research study, know that there are certain limitations to what you are testing or possible research results. There are different types that researchers may encounter, and they all have unique characteristics, such as:

1. Research Design Limitations

Certain restrictions on your research or available procedures may affect your final results or research outputs. You may have formulated research goals and objectives too broadly. However, this can help you understand how you can narrow down the formulation of research goals and objectives, thereby increasing the focus of your study.

2. Impact Limitations

Even if your research has excellent statistics and a strong design, it can suffer from the influence of the following factors:

• Presence of increasing findings as researched
• Being population specific
• A strong regional focus.

3. Data or statistical limitations

In some cases, it is impossible to collect sufficient data for research or very difficult to get access to the data. This could lead to incomplete conclusion to your study. Moreover, this insufficiency in data could be the outcome of your study design. The unclear, shabby research outline could produce more problems in interpreting your findings.

How to Correctly Structure Your Research Limitations?

There are strict guidelines for narrowing down research questions, wherein you could justify and explain potential weaknesses of your academic paper. You could go through these basic steps to get a well-structured clarity of research limitations:

• Declare that you wish to identify your limitations of research and explain their importance,
• Provide the necessary depth, explain their nature, and justify your study choices.
• Write how you are suggesting that it is possible to overcome them in the future.

In this section, your readers will see that you are aware of the potential weaknesses in your business, understand them and offer effective solutions, and it will positively strengthen your article as you clarify all limitations of research to your target audience.

Know that you cannot be perfect and there is no individual without flaws. You could use the limitations of research as a great opportunity to take on a new challenge and improve the future of research. In a typical academic paper, research limitations may relate to:

1. Formulating your goals and objectives

If you formulate goals and objectives too broadly, your work will have some shortcomings. In this case, specify effective methods or ways to narrow down the formula of goals and aim to increase your level of study focus.

2. Application of your data collection methods in research

If you do not have experience in primary data collection, there is a risk that there will be flaws in the implementation of your methods. It is necessary to accept this, and learn and educate yourself to understand data collection methods.

3. Sample sizes

This depends on the nature of problem you choose. Sample size is of a greater importance in quantitative studies as opposed to qualitative ones. If your sample size is too small, statistical tests cannot identify significant relationships or connections within a given data set.

You could point out that other researchers should base the same study on a larger sample size to get more accurate results.

4. The absence of previous studies in the field you have chosen

Writing a literature review is an important step in any scientific study because it helps researchers determine the scope of current work in the chosen field. It is a major foundation for any researcher who must use them to achieve a set of specific goals or objectives.

However, if you are focused on the most current and evolving research problem or a very narrow research problem, there may be very little prior research on your topic. For example, if you chose to explore the role of Bitcoin as the currency of the future, you may not find tons of scientific papers addressing the research problem as Bitcoins are only a new phenomenon.

It is important that you learn to identify research limitations examples at each step. Whatever field you choose, feel free to add the shortcoming of your work. This is mainly because you do not have many years of experience writing scientific papers or completing complex work. Therefore, the depth and scope of your discussions may be compromised at different levels compared to academics with a lot of expertise. Include specific points from limitations of research. Use them as suggestions for the future.

Have you ever faced a challenge of writing the limitations of research study in your paper? How did you overcome it? What ways did you follow? Were they beneficial? Let us know in the comments below!

Frequently Asked Questions

Setting limitations in our study helps to clarify the outcomes drawn from our research and enhance understanding of the subject. Moreover, it shows that the author has investigated all the weaknesses in the study.

Scope is the range and limitations of a research project which are set to define the boundaries of a project. Limitations are the impacts on the overall study due to the constraints on the research design.

Limitation in research is an impact of a constraint on the research design in the overall study. They are the flaws or weaknesses in the study, which may influence the outcome of the research.

1. Limitations in research can be written as follows: Formulate your goals and objectives 2. Analyze the chosen data collection method and the sample sizes 3. Identify your limitations of research and explain their importance 4. Provide the necessary depth, explain their nature, and justify your study choices 5. Write how you are suggesting that it is possible to overcome them in the future

Excellent article ,,,it has helped me big

This is very helpful information. It has given me an insight on how to go about my study limitations.

Good comments and helpful

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Diversity and Inclusion
• Trending Now

The Silent Struggle: Confronting gender bias in science funding

In the 1990s, Dr. Katalin Kariko’s pioneering mRNA research seemed destined for obscurity, doomed by…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Addressing Barriers in Academia: Navigating unconscious biases in the Ph.D. journey

In the journey of academia, a Ph.D. marks a transitional phase, like that of a…

• Manuscripts & Grants
• Reporting Research

Unraveling Research Population and Sample: Understanding their role in statistical inference

Research population and sample serve as the cornerstones of any scientific inquiry. They hold the…

• Manuscript Preparation
• Publishing Research

Research Problem Statement — Find out how to write an impactful one!

What Is a Research Problem Statement? A research problem statement is a clear, concise, and…

How to Develop a Good Research Question? — Types & Examples

5 Effective Ways to Avoid Ghostwriting for Busy Researchers

Top 5 Key Differences Between Methods and Methodology

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

What should universities' stance be on AI tools in research and academic writing?

• Privacy Policy

Buy Me a Coffee

Home » Limitations in Research – Types, Examples and Writing Guide

Limitations in Research – Types, Examples and Writing Guide

Table of Contents

Limitations in Research

Limitations in research refer to the factors that may affect the results, conclusions , and generalizability of a study. These limitations can arise from various sources, such as the design of the study, the sampling methods used, the measurement tools employed, and the limitations of the data analysis techniques.

Types of Limitations in Research

Types of Limitations in Research are as follows:

Sample Size Limitations

This refers to the size of the group of people or subjects that are being studied. If the sample size is too small, then the results may not be representative of the population being studied. This can lead to a lack of generalizability of the results.

Time Limitations

Time limitations can be a constraint on the research process . This could mean that the study is unable to be conducted for a long enough period of time to observe the long-term effects of an intervention, or to collect enough data to draw accurate conclusions.

Selection Bias

This refers to a type of bias that can occur when the selection of participants in a study is not random. This can lead to a biased sample that is not representative of the population being studied.

Confounding Variables

Confounding variables are factors that can influence the outcome of a study, but are not being measured or controlled for. These can lead to inaccurate conclusions or a lack of clarity in the results.

Measurement Error

This refers to inaccuracies in the measurement of variables, such as using a faulty instrument or scale. This can lead to inaccurate results or a lack of validity in the study.

Ethical Limitations

Ethical limitations refer to the ethical constraints placed on research studies. For example, certain studies may not be allowed to be conducted due to ethical concerns, such as studies that involve harm to participants.

Examples of Limitations in Research

Some Examples of Limitations in Research are as follows:

Research Title: “The Effectiveness of Machine Learning Algorithms in Predicting Customer Behavior”

Limitations:

• The study only considered a limited number of machine learning algorithms and did not explore the effectiveness of other algorithms.
• The study used a specific dataset, which may not be representative of all customer behaviors or demographics.
• The study did not consider the potential ethical implications of using machine learning algorithms in predicting customer behavior.

Research Title: “The Impact of Online Learning on Student Performance in Computer Science Courses”

• The study was conducted during the COVID-19 pandemic, which may have affected the results due to the unique circumstances of remote learning.
• The study only included students from a single university, which may limit the generalizability of the findings to other institutions.
• The study did not consider the impact of individual differences, such as prior knowledge or motivation, on student performance in online learning environments.

Research Title: “The Effect of Gamification on User Engagement in Mobile Health Applications”

• The study only tested a specific gamification strategy and did not explore the effectiveness of other gamification techniques.
• The study relied on self-reported measures of user engagement, which may be subject to social desirability bias or measurement errors.
• The study only included a specific demographic group (e.g., young adults) and may not be generalizable to other populations with different preferences or needs.

How to Write Limitations in Research

When writing about the limitations of a research study, it is important to be honest and clear about the potential weaknesses of your work. Here are some tips for writing about limitations in research:

• Identify the limitations: Start by identifying the potential limitations of your research. These may include sample size, selection bias, measurement error, or other issues that could affect the validity and reliability of your findings.
• Be honest and objective: When describing the limitations of your research, be honest and objective. Do not try to minimize or downplay the limitations, but also do not exaggerate them. Be clear and concise in your description of the limitations.
• Provide context: It is important to provide context for the limitations of your research. For example, if your sample size was small, explain why this was the case and how it may have affected your results. Providing context can help readers understand the limitations in a broader context.
• Discuss implications : Discuss the implications of the limitations for your research findings. For example, if there was a selection bias in your sample, explain how this may have affected the generalizability of your findings. This can help readers understand the limitations in terms of their impact on the overall validity of your research.
• Provide suggestions for future research : Finally, provide suggestions for future research that can address the limitations of your study. This can help readers understand how your research fits into the broader field and can provide a roadmap for future studies.

Purpose of Limitations in Research

There are several purposes of limitations in research. Here are some of the most important ones:

• To acknowledge the boundaries of the study : Limitations help to define the scope of the research project and set realistic expectations for the findings. They can help to clarify what the study is not intended to address.
• To identify potential sources of bias: Limitations can help researchers identify potential sources of bias in their research design, data collection, or analysis. This can help to improve the validity and reliability of the findings.
• To provide opportunities for future research: Limitations can highlight areas for future research and suggest avenues for further exploration. This can help to advance knowledge in a particular field.
• To demonstrate transparency and accountability: By acknowledging the limitations of their research, researchers can demonstrate transparency and accountability to their readers, peers, and funders. This can help to build trust and credibility in the research community.
• To encourage critical thinking: Limitations can encourage readers to critically evaluate the study’s findings and consider alternative explanations or interpretations. This can help to promote a more nuanced and sophisticated understanding of the topic under investigation.

When to Write Limitations in Research

Limitations should be included in research when they help to provide a more complete understanding of the study’s results and implications. A limitation is any factor that could potentially impact the accuracy, reliability, or generalizability of the study’s findings.

It is important to identify and discuss limitations in research because doing so helps to ensure that the results are interpreted appropriately and that any conclusions drawn are supported by the available evidence. Limitations can also suggest areas for future research, highlight potential biases or confounding factors that may have affected the results, and provide context for the study’s findings.

Generally, limitations should be discussed in the conclusion section of a research paper or thesis, although they may also be mentioned in other sections, such as the introduction or methods. The specific limitations that are discussed will depend on the nature of the study, the research question being investigated, and the data that was collected.

Examples of limitations that might be discussed in research include sample size limitations, data collection methods, the validity and reliability of measures used, and potential biases or confounding factors that could have affected the results. It is important to note that limitations should not be used as a justification for poor research design or methodology, but rather as a way to enhance the understanding and interpretation of the study’s findings.

Importance of Limitations in Research

Here are some reasons why limitations are important in research:

• Enhances the credibility of research: Limitations highlight the potential weaknesses and threats to validity, which helps readers to understand the scope and boundaries of the study. This improves the credibility of research by acknowledging its limitations and providing a clear picture of what can and cannot be concluded from the study.
• Facilitates replication: By highlighting the limitations, researchers can provide detailed information about the study’s methodology, data collection, and analysis. This information helps other researchers to replicate the study and test the validity of the findings, which enhances the reliability of research.
• Guides future research : Limitations provide insights into areas for future research by identifying gaps or areas that require further investigation. This can help researchers to design more comprehensive and effective studies that build on existing knowledge.
• Provides a balanced view: Limitations help to provide a balanced view of the research by highlighting both strengths and weaknesses. This ensures that readers have a clear understanding of the study’s limitations and can make informed decisions about the generalizability and applicability of the findings.

Advantages of Limitations in Research

Here are some potential advantages of limitations in research:

• Focus : Limitations can help researchers focus their study on a specific area or population, which can make the research more relevant and useful.
• Realism : Limitations can make a study more realistic by reflecting the practical constraints and challenges of conducting research in the real world.
• Innovation : Limitations can spur researchers to be more innovative and creative in their research design and methodology, as they search for ways to work around the limitations.
• Rigor : Limitations can actually increase the rigor and credibility of a study, as researchers are forced to carefully consider the potential sources of bias and error, and address them to the best of their abilities.
• Generalizability : Limitations can actually improve the generalizability of a study by ensuring that it is not overly focused on a specific sample or situation, and that the results can be applied more broadly.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Understanding Commonly Encountered Limitations in Clinical Research: An Emergency Medicine Resident’s Perspective

Associated data, introduction.

The breadth of emergency medicine and the rapid growth in relevant research makes an ability to assess new research findings particularly important for emergency physicians. Improvements in the treatment of acute myocardial infarction, evolution of thrombolytic use in acute stroke, and the demise of military antishock trousers for traumatic shock provide examples of the dynamic relationship between emergency medicine research and clinical practice. The purpose of this article is to provide an overview of common research limitations and flaws relevant to emergency medicine. We explain and provide published examples of problems related to external validity, experimenter bias, publication bias, straw man comparisons, incorporation bias, randomization, composite outcomes, clinical importance versus statistical significance, and disease-oriented versus patient-oriented outcomes.

For residents, familiarity with these concepts will allow for better interpretation of evidence-based lectures and improved understanding and participation during journal clubs. For those who learn best from review articles, textbooks, or other summaries of primary literature, awareness of these issues is needed to understand critiques raised by reviewers. For all readers, knowledge of these commonly encountered methodological problems will improve the emergency provider’s ability to determine whether and how new scientific developments serve to inform clinical practice. By necessity, this article is only a starting point for learning about these subjects, many of which are complex. Readers are encouraged to refer to questions and answers from the Annals of Emergency Medicine Journal Club series. Available topics from this series are described in Appendix E1 , available online at http://www.annemergmed.com .

EXTERNAL VALIDITY

External validity is the generalizability of a study’s conclusions beyond the specific sample examined. In some studies, the study patients will be very similar to relevant clinical populations; in other studies, the lack of generalizability will render the results useless to the reader. The applicability of study findings may differ among geographic regions or specialties, or even among providers with different skills and experience practicing in the same emergency department (ED). Common threats to external validity include studies examining restricted or atypical demographic groups, studies performed at different sites of care (eg, specialty clinic versus community ED versus academic ED), and studies conducted in countries or settings in which available resources differ from the setting in which the reader practices. Issues related to external validity are particularly important in emergency medicine because the patient population is inherently heterogeneous in terms of demographics, acuity, comorbidities, and referral source.

A recent investigation into the need for lumbar puncture after head computed tomography (CT) for evaluation of subarachnoid hemorrhage illustrates the importance of considering external validity. 1 In this study, head CT within 5 days of the onset of headache was found to be 100% sensitive for subarachnoid hemorrhage. The authors used this finding to suggest that lumbar puncture is no longer necessary to conclusively rule out subarachnoid hemorrhage in patients presenting with thunderclap headache. However, the study sample consisted of patients referred to a specialty neurosurgical center for evaluation of known or possible subarachnoid hemorrhage. Patients with suspected subarachnoid hemorrhage referred to a neurosurgical center for evaluation are arguably a very different population than undifferentiated patients initially presenting to an ED. If, for example, referred patients with subarachnoid hemorrhage tend to have larger bleeding episodes than ED patients, referred patients would also be more likely to have abnormal CT scan results. In a neurosurgery referral center population, perhaps it is true that 100% of patients with subarachnoid hemorrhage will have a positive CT result and there is no additional value to performing a lumbar puncture. However, we should be cautious about applying this conclusion to ED patients with a sudden-onset headache.

Whereas external validity depends on whether study conclusions are generalizable to other populations, internal validity depends on whether study conclusions are valid within the study sample. There are many potential threats to internal validity; several are described below. When a study is designed, there is often a tradeoff between internal and external validity. A highly controlled study of a carefully specified patient group will tend to have good internal validity, but this can limit its external validity.

EXPERIMENTER BIAS

Experimenter bias is introduced by study investigators because of the inability of investigators to be completely objective. Experimenter bias may be conscious or unconscious and may influence the study through choices about study design, 2 implementation, reporting and interpretation of results, 3 or even a decision to publish results at all. 4 Few investigations are initiated without preconceptions on the part of the investigators about possible outcomes. For clinical scientists, the desire for academic success and the associated desire to publish is a source of bias. Scientists may also be influenced by the desire to provide further evidence to support their previous work and to obtain or maintain grant funding. Investigators conducting industry-sponsored studies may be influenced by financial incentives. Financial conflicts of interest are rampant in both original research articles 5 and clinical guidelines. 6 Most journals require authors to disclose financial conflicts of interest. These disclosures do not eliminate the possibility of experimenter bias but can at least inform readers about the presence of external influences on decisions made by investigators. Unfortunately, the presence of experimenter bias is difficult to determine with certainty.

Nesiritide, recombinant brain-type natriuretic peptide, provides an example of possible experimenter bias. The Vasodilation in the Management of Acute Congestive Heart Failure (VMAC) study was a large randomized controlled trial comparing nesiritide with nitroglycerin for acutely decompensated congestive heart failure. 7 The study was funded by Scios Inc, the manufacturers of nesiritide. Additionally, the study was designed, implemented, and analyzed by a steering committee appointed by Scios. Multiple investigators, including the principal investigator, received consulting fees from Scios. In 2000, after the completion of the VMAC study but before peer-reviewed publication of study results, Scios estimated that peak sales of nesiritide would reach $200 to $300 million per year. This was a strong incentive for Scios to ensure the success of its product, particularly given that total company revenues that year were just $12.7 million. 8 The initial article detailing VMAC results endorsed wide use of the drug, stating “nesiritide … is a useful addition to initial therapy of patients hospitalized with acutely decompensated CHF [congestive heart failure],” 7 despite data showing a trend toward increased mortality in the nesiritide group. 9

PUBLICATION BIAS

Interventional studies that show a positive effect from the intervention are more likely to be published than those that do not, resulting in a pro-intervention bias within the literature base. 4 Publication bias is especially important to keep in mind when meta-analyses are evaluated because it is more difficult to obtain data from unpublished trials than published trials when constructing a meta-analysis. Additionally, meta-analyses frequently consist of numerous small or methodologically challenged studies, types of studies that are likely to be published only if the results are positive. A funnel plot can show evidence of publication bias in a meta-analysis. The funnel is constructed by plotting risk ratio or another measure of treatment effect on the x axis and standard error or another measure of precision on the y axis. By convention, the y axis is arranged so that larger, more precise studies are at the top, with smaller imprecise studies at the bottom. The more precise studies should cluster tightly around the true effect value on the x axis because they will tend to provide more accurate estimates of the effect. Less precise studies will have a wide variety of effect estimates and will not cluster as tightly, creating the funnel shape. When a subset of these smaller, imprecise studies is not published, the result is a skewed distribution of effect estimates among the smaller studies, creating asymmetry in the funnel plot. Funnel plots lack both sensitivity and specificity for diagnosing publication bias because they are often limited by small numbers of available studies, systematic differences in study methodology among studies of varying size, and subjectivity in interpretation. 10

Evidence of publication bias is found in studies of intravenous magnesium to decrease mortality in the setting of acute myocardial infarction. 11 Two meta-analyses, based on relatively small studies, suggested that magnesium use conferred a survival benefit in this setting. 12 , 13 A large randomized controlled trial involving more than 58,000 patients contradicted these findings by showing no benefit from magnesium. 14 A funnel plot shows evidence of publication bias present in the studies used for the initial meta-analyses ( Figure ). 11

Funnel plot for magnesium trials. Dots represent treatment effects from individual trials. The diagonal lines represent the expected 95% confidence intervals surrounding the pooled treatment effect. The absence of trials in the lower right corner of the figure suggests publication bias. Adapted from Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol . 2001;54: 1048. Figure 1 used with permission from Elsevier.

STRAW MAN COMPARISON

A comparison between an experimental group and a control group that receives substandard therapy is a straw man comparison. Unfortunately, this technique is frequently used to provide support for novel therapeutic agents or new applications of existing agents. It is easier to find evidence of equivalence or superiority of a novel intervention when the control group receives inadequate therapy. Relevant considerations include the choice of control-group therapy, as well as dose and frequency.

Straw man comparators were used in 2 manufacturer-sponsored studies comparing lumiracoxib, a selective Cox-2 inhibitor, and ibuprofen. 15 , 16 The first of these claimed to test the analgesic efficacy of lumiracoxib. It compared the maximum daily dose of lumiracoxib with a single dose of 400 mg of ibuprofen for pain control after tooth extraction. 15 The study concluded that lumiracoxib was superior to ibuprofen. However, one sixth of the standard 24-hour dose of ibuprofen was used as the comparator. A second study involving these 2 drugs, also sponsored by Novartis, compared complications related to gastrointestinal ulcers between treatment groups. 16 Patients older than 50 years and with osteoarthritis received the maximum daily dose of lumiracoxib or 800 mg ibuprofen 3 times per day for 1 year. They concluded that the risk of complications from upper gastrointestinal tract ulcers was lower in the lumiracoxib group. This ibuprofen dose is much higher than the dose used when the primary outcome was pain control. In both of these cases, dosing in the active control (ibuprofen) group was designed to magnify any advantage of lumiracoxib compared with ibuprofen.

The use of straw man comparators is always misleading, at times unethical, and in some instances illegal. An extreme example of this occurred in Nigeria during a 1996 meningitis epidemic. Pfizer compared trovafloxacin, a novel fluoroquinolone, with ceftriaxone in children with bacterial meningitis. Although the results of this study have not been published in a peer-reviewed journal, details were documented by the Washington Post . 17 Their investigation revealed that most of the children in the ceftriaxone group received intramuscular doses of 33 mg/kg per day, far below the recommended intravenous dosing of 100 mg/kg per day for bacterial meningitis. A statement from Pfizer justified this discrepancy by stating that the change in dosing was done “to diminish the significant pain resulting from the intramuscular injection.” 18 Conveniently for Pfizer, this decision also increased the likelihood that trovafloxacin would outperform ceftriaxone. In 2009, Pfizer reached a financial settlement to compensate families involved in the study. 19

INCORPORATION BIAS

Incorporation bias occurs in studies of diagnostic tests in which one of the tests being examined and the criterion standard (aka reference standard) are not independent. This commonly occurs when 2 diagnostic tests are compared and one of these tests is also used to define the “correct” answer. The test that is used to define the reference standard will always outperform the other test.

Incorporation bias is present in a retrospective chart review of blunt trauma victims who had both cervical CT and radiography to evaluate for cervical spine fracture. 20 The authors reported that CT was 100% sensitive for the detection of these injuries in the included cohort, whereas radiography was 65% sensitive. However, CT results were used as the criterion standard. This approach ignores the possibility that some of the injuries observed on CT were artifacts or even that the CT missed fractures observed on radiographs. The study used circular reasoning: the authors used CT as the criterion standard because they believed it was superior to radiography, and they then used their results to “prove” this claim. As designed, it was inevitable that the authors would reach the conclusion that “[c]omputed tomography of the cervical spine should replace cervical spine radiographs ….” 20

UNSUCCESSFUL RANDOMIZATION

Randomization is the process by which study participants are allocated to particular treatment groups by a random process such as a coin flip or computerized random-number generator. The purpose of randomization is to ensure that aside from the experimental intervention, study groups are as similar as possible. If randomization is unsuccessful or is performed poorly, study results may be due to the presence of differences between the 2 groups other than the intervention, which is more likely to happen in small studies than in large studies. Stratified randomization helps prevent this by balancing rates of key characteristics among study groups. In addition to comparing the characteristics of the treatment and control groups provided by study authors, one should also consider whether the groups might differ in ways not described in the article.

The European Cooperative Acute Stroke Study III is a recently published randomized trial of thrombolytic therapy (alteplase) versus placebo administered between 3.0 and 4.5 hours after the onset of stroke symptoms. 21 The mean pretreatment National Institutes of Health Stroke Scale (NIHSS) score in the alteplase group was 10.7 compared with 11.6 in the placebo group, suggesting that baseline stroke severity was modestly worse in the placebo group. Additionally, 7.7% of patients in the alteplase group and 14.1% in the placebo group had a history of stroke, a factor that has been associated with poor functional outcomes. More significant is that only 5% of patients in the alteplase group had initial NIHSS scores greater than 20 compared with 10% of patients in the placebo group. 22 These figures were not reported in the initial European Cooperative Acute Stroke Study III article; however, they are of critical importance. Compared with other stroke patients, those with NIHSS scores above 20 have very poor outcomes regardless of any intervention, and patients with scores below 10 usually do well regardless of any intervention. 23 Together, these baseline imbalances in stroke severity between study groups potentially account for the entire 7.2% absolute difference in favorable functional outcome reported in this study. Randomization can be performed in ways that limit this problem. For example, a stratified randomization scheme using NIHSS scores to account for stroke severity could have been used to balance the allocation of patients with mild, moderate, and severe strokes among the alteplase and placebo groups.

COMPOSITE OUTCOMES

Composite outcomes are created by combining multiple endpoints into a single common outcome measure. Investigators sometimes use this technique because finding a statistically significant difference between outcomes is easier when outcomes are common. Composite outcomes can make the interpretation of results difficult because the individual factors making up a given composite outcome are frequently not equivalent, clouding the clinical significance of the composite measure. The problem of composite endpoints can be at least partially addressed by comparing the frequency of the individual components if these results are provided by the authors. Others have recommended specific questions to ask when assessing the usefulness of composite outcomes ( Table ). 24

Questions used to assess for limitations commonly encountered in clinical research. *

The Clopidogrel in Unstable Angina to Prevent Recurrent Events Trial (CURE) was an industry-sponsored study comparing clopidogrel with placebo in patients with non–ST-elevation myocardial infarction. 25 The primary outcome was a composite of death from a cardiovascular cause, stroke, or recurrent nonfatal myocardial infarction. Recurrent myocardial infarction was defined as an increase in cardiac biomarker levels to at least double the upper limit of normal, whether or not this increase was considered clinically significant. The difference in composite outcome was largely driven by increased recurrent myocardial infarction in the placebo group (6.7% versus 5.2%) rather than by a nonsignificant absolute reduction in death of 0.4% in the treatment group. In fact, recurrent myocardial infarction was the only component of the composite outcome that reached individual statistical significance. Reporting that “clopidogrel significantly reduces the risk of the composite outcome of death from cardiovascular causes, nonfatal myocardial infarction, or stroke” 25 is more likely to promote clopidogrel use than stating “clopidogrel significantly reduces the risk of an asymptomatic rise in cardiac biomarkers, without a significant reduction in mortality or recurrent stroke.”

CLINICAL IMPORTANCE VERSUS STATISTICAL SIGNIFICANCE

Statistical significance is reached when study results justify the rejection of the study’s null hypothesis with a predetermined level of probability. Traditionally, but somewhat arbitrarily, if the probability of obtaining an observed result or one more extreme by chance alone is less than 5% (ie, P < .05), then a result is judged statistically significant. The clinical importance of a finding is influenced by the importance of the outcome measured, the magnitude of differences observed, and the existence of alternative therapies. Findings that are clinically important but not statistically significant ought to be repeated with a larger sample; findings that are statistically significant but not clinically important are irrelevant. A growing viewpoint in the research community, termed Bayesian inference, represents an alternative to frequentist statistics. Rather than focusing on whether a particular finding is mathematically significant, a Bayesian approach seeks to determine the truth of a particular finding by incorporating experimentally observed evidence together with the prior probability of a given hypothesis based on previous research. 26 , 27

The use of dexamethasone in patients with migraine headaches provides an example of possible discordance between clinical importance and statistical significance. A randomized controlled trial was performed in which 130 patients with migraine headache received 15 mg dexamethasone or placebo. 28 Within the 48 hours after ED discharge, 32% of patients in the placebo arm developed a severe recurrent headache compared with 22% of patients in the dexamethasone group. The odds ratio in favor of dexamethasone was 0.6, with a 95% confidence interval of 0.3 to 1.3. Although this result is not statistically significant, a 10% difference in headache relapse is clinically important and justifies further investigation. A subsequent, adequately powered meta-analysis demonstrated a statistically significant improvement in relapse rate of similar magnitude. 29

DISEASE-ORIENTED (INTERMEDIATE) VERSUS PATIENT-ORIENTED OUTCOMES

Clinical decisions should be based on outcomes that matter to patients. Disease-oriented outcomes describe the presence or severity of a particular disease. Examples of disease-oriented outcomes include laboratory values, ECG findings, and abnormalities on imaging studies. Studies using patient-oriented outcomes, on the other hand, measure survival or some aspect of quality of life such as disability, pain, out-of-pocket costs, time in a hospital, or time away from work. Disease-oriented outcomes are often favored in clinical research because they are easier to measure. Examining disease-oriented outcomes may be a necessary step in understanding pharmacology or pathophysiology, but ideally, clinical decisions should be supported by studies demonstrating an effect on patient-oriented outcomes.

Controversy has surrounded the association between etomidate use and adrenal insufficiency. A recent randomized controlled trial compared etomidate with ketamine for use in rapid sequence intubation. 30 Etomidate was associated with a significant increase in adrenal insufficiency as defined by failure to respond to a cortisol stimulation test, or a random cortisol level below 276 nmol/L measured within 48 hours after tracheal intubation. Clearly, an individual patient will not know or care whether his or her serum cortisol level is slightly above or below 276 nmol/L. As addressed by the authors, this study did not demonstrate any significant difference in mortality or mechanical ventilation–free days, both of which are patient-oriented outcomes of greater clinical importance than serum cortisol test results.

Military antishock trousers provide a more dramatic example of how disease-oriented outcomes and patient-oriented outcomes can be at odds. This device, which encases the lower half of the body in high-pressure air bladders, became a staple of emergency medical services care for the treatment of hypovolemic shock in the United States during the 1970s. 31 Their use was justified because they increased blood pressure for hypotensive trauma patients. 32 Hypotension, however, is a disease-oriented outcome that is important only because of its assumed relationship to mortality or other patient-oriented outcomes. In fact, a subsequent meta-analysis suggested that military antishock trousers use was associated with increased mortality. 33 The decision to use the trousers according to their effects on blood pressure cost money and probably cost lives.

CONCLUSIONS

These examples are intended to illustrate limitations commonly encountered in research relevant to the practice of emergency medicine. No study is perfect. Even studies that include significant biases can be valuable if interpreted with an awareness of relevant limitations. Ideally, the discussion section of an original research article will both identify these limitations and include a Bayesian interpretation of the findings, in which the meaning of the new results is added to existing literature on the subject to obtain a poststudy result. If this is not done by the study authors, this work is left to the reader and to those who write article summaries and reviews. This method of integrating new data with existing information is particularly important in light of the constantly expanding medical knowledge base. A recent analysis of interventional studies published in top medical journals between 1990 and 2003 found that by 2005, a third had been contradicted or faced significant challenges about the magnitude of initially reported effect sizes. 34 These findings highlight the need for clinicians to question both new and established research results ( Table ). Ultimately, clinicians, not researchers, make decisions that affect patients. Now more than ever, good clinical care should be rooted in the process of judging new research according to its scientific merit, considering the relevance of new data to one’s own patient population, and ensuring that the outcomes we care about are the ones that matter for our patients.

Supplementary Material

Acknowledgments.

Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org ). The authors have stated that no such relationships exist.

What limitations are reported in short articles in social and personality psychology?

Affiliations.

• 1 Melbourne School of Psychological Sciences, University of Melbourne.
• 2 Department of Psychology, University of California, Davis.
• PMID: 36996169
• DOI: 10.1037/pspp0000458

Every research project has limitations. The limitations that authors acknowledge in their articles offer a glimpse into some of the concerns that occupy a field's attention. We examine the types of limitations authors discuss in their published articles by categorizing them according to the four validities framework and investigate whether the field's attention to each of the four validities has shifted from 2010 to 2020. We selected one journal in social and personality psychology ( Social Psychological and Personality Science; SPPS ), the subfield most in the crosshairs of psychology's replication crisis. We sampled 440 articles (with half of those articles containing a subsection explicitly addressing limitations), and we identified and categorized 831 limitations across the 440 articles. Articles with limitations sections reported more limitations than those without (avg. 2.6 vs. 1.2 limitations per article). Threats to external validity were the most common type of reported limitation (est. 52% of articles), and threats to statistical conclusion validity were the least common (est. 17% of articles). Authors reported slightly more limitations over time. Despite the extensive attention paid to statistical conclusion validity in the scientific discourse throughout psychology's credibility revolution, our results suggest that concerns about statistics-related issues were not reflected in social and personality psychologists' reported limitations. The high prevalence of limitations concerning external validity might suggest it is time that we improve our practices in this area, rather than apologizing for these limitations after the fact. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

• Personality Disorders*
• Personality*
• Research Design

Grants and funding

• National Science Foundation

• Share full article

Advertisement

Supported by

Is Intermittent Fasting Bad for Your Heart? Here’s What We Know.

By Alice Callahan

You may have seen the headlines: “Intermittent fasting linked to 91 percent increase in risk of death from heart disease”; “The intermittent fasting trend may pose risks to your heart.”

The news came from an abstract presented Monday at an American Heart Association conference in Chicago. The study has not yet been published in a peer-reviewed journal, and experts cautioned that it had many limitations. Here’s what we know.

The Background

Intermittent fasting involves cycling between eating and fasting for specific periods of time. A common approach, for example, is to eat only within an eight-hour window each day, said Krista Varady, a professor of nutrition at the University of Illinois Chicago.

Several short-term trials have suggested that this eating style can lead to some weight loss and may lower blood pressure and improve blood sugar control in certain people, she said.

But the longest intermittent fasting trial lasted only one year , said Victor Wenze Zhong, lead author of the new study and an epidemiologist at Shanghai Jiao Tong University School of Medicine in China. His aim, he said, was to look at longer term health.

The Research

The new study included more than 20,000 adults from the United States. The participants completed two interviews, less than two weeks apart, about what time they ate on the previous day. The researchers then calculated the participants’ average eating windows and assumed it was their typical schedule for the rest of the study, Dr. Zhong said. The participants were followed for an average of eight years.

During that time, the participants who limited their eating to eight hours in a day had a 91 percent greater chance of dying from cardiovascular disease than those who ate over a 12- to 16-hour time frame, the researchers reported.

But there were just 414 people in the eight-hour eating group, Dr. Zhong said. And they tended to be younger and less educated; have lower income and less access to food; and be more likely to smoke than the other participants.

The researchers accounted for these factors in their analysis, Dr. Zhong said. But the study did not show that this style of eating caused deaths from cardiovascular disease, only that the two were linked.

The Limitations

Since the study has not been published or peer-reviewed, it’s challenging to fully evaluate it, Dr. Varady said.

A “major limitation” is that they used just two reports to accurately represent people’s typical eating pattern, Dr. Varady said; and the study did not seem to evaluate what kinds of foods people ate.

Dr. Dariush Mozaffarian, a cardiologist and professor of medicine at Tufts University, called the study “very problematic.” The eight-hour eating group may have included many people who were very busy, or faced other challenges that forced them to miss meals or eat erratically, he said.

The group also could have included people who were already in poor health — those with eating disorders or illness that reduced their appetite, for instance, which may have resulted in them eating during a shorter window, said Satchidananda Panda, a professor at the Salk Institute for Biological Studies in San Diego.

And if intermittent fasting is really harmful, it’s not clear why that might be. Dr. Zhong said that his study was not designed to answer that question.

What’s Next

More research is needed to evaluate the long term health effects of intermittent fasting, Dr. Zhong said.

Intermittent fasting isn’t a good fit for everyone, said Dr. Pam Taub, a cardiologist at the University of California, San Diego. But many of her patients have enjoyed its benefits, like reduced cholesterol levels.

Now, her patients are “confused and scared” by the headlines they’re reading, Dr. Taub said. But she won’t recommend that they change anything based on this study, she said, adding that people should always talk with their doctor before shifting their diet or lifestyle.

An earlier version of this story misrepresented the way researchers collected diet information with the study participants. It was via two interviews, not two questionnaires.

How we handle corrections

Alice Callahan is a Times reporter covering nutrition and health. She has a Ph.D. in nutrition from the University of California, Davis. More about Alice Callahan

A Guide to Better Nutrition

How much salt is too much? Should I cut back ? We asked experts these and other questions about sodium.

Patients were told for years that cutting calories would ease the symptoms of polycystic ovary syndrome. But research suggests dieting may not help at all .

We asked a nutrition expert how she keeps up healthy habits without stressing about food. Here are seven tips  she shared for maintaining that balance.

There are many people who want to lose a few pounds for whom weight loss drugs are not the right choice. Is old-fashioned dieting a good option ?

Salmon is good for you, but choosing the right type to eat isn’t so easy. Here are answers to all your questions about this nutritional powerhouse .

Read these books to shift into a healthier way of thinking about food .

Sign up for Well’s Mediterranean diet week : Each day, we’ll send guidance and recipes to help make 2024 your most nourishing year yet.

This paper is in the following e-collection/theme issue:

Published on 29.3.2024 in Vol 26 (2024)

Telehealth Care Through Internet Hospitals in China: Qualitative Interview Study of Physicians’ Views on Access, Expectations, and Communication

Authors of this article:

Original Paper

• Yuqiong Zhong 1, 2 , Mphil   ; 
• Jessica Hahne 3 , MA, MPH   ; 
• Xiaomin Wang 4, 5 , PhD   ; 
• Xuxi Wang 1 , Bphil   ; 
• Ying Wu 1 , MPhil   ; 
• Xin Zhang 2, 6 * , MD, PhD   ; 
• Xing Liu 6, 7 * , PhD  

1 School of Humanities, Central South University, Changsha, China

2 Xiangya Hospital, Central South University, Changsha, China

3 Department of Psychological & Brain Sciences, Washington University in St Louis, St Louis, MO, United States

4 Center for Clinical Pharmacology, The Third Xiangya Hospital of Central South University, Changsha, China

5 Center for Medical Ethics, Central South University, Changsha, China

6 Medical Humanities Research Center, Central South University, Changsha, China

7 Office of International Cooperation and Exchanges, Xiangya Hospital, Central South University, Changsha, China

*these authors contributed equally

Corresponding Author:

Xing Liu, PhD

Office of International Cooperation and Exchanges

Xiangya Hospital, Central South University

No 87 Xiangya Road, Kaifu District

Changsha, 410008

Phone: 86 18229765509

Email: [email protected]

Background: Internet hospitals in China are an emerging medical service model similar to other telehealth models used worldwide. Internet hospitals are currently in a stage of rapid development, giving rise to a series of new opportunities and challenges for patient care. Little research has examined the views of chronic disease physicians regarding internet hospitals in China.

Objective: We aimed to explore the experience and views of chronic disease physicians at 3 tertiary hospitals in Changsha, China, regarding opportunities and challenges in internet hospital care.

Methods: We conducted semistructured qualitative interviews with physicians (n=26) who had experience working in internet hospitals affiliated with chronic disease departments in 3 tertiary hospitals in Changsha, Hunan province, south central China. Interviews were transcribed verbatim and analyzed by content analysis using NVivo software (version 11; Lumivero).

Results: Physicians emphasized that internet hospitals expand opportunities to conduct follow-up care and health education for patients with chronic illnesses. However, physicians described disparities in access for particular groups of patients, such as patients who are older, patients with lower education levels, patients with limited internet or technology access, and rural patients. Physicians also perceived a gap between patients’ expectations and the reality of limitations regarding both physicians’ availability and the scope of services offered by internet hospitals, which raised challenges for doctor-patient boundaries and trust. Physicians noted challenges in doctor-patient communication related to comprehension and informed consent in internet hospital care.

Conclusions: This study explored the experience and views of physicians in 3 tertiary hospitals in Changsha, China, regarding access to care, patients’ expectations versus the reality of services, and doctor-patient communication in internet hospital care. Findings from this study highlight the need for physician training in telehealth communication skills, legislation regulating informed consent in telehealth care, public education clarifying the scope of internet hospital services, and design of internet hospitals that is informed by the needs of patient groups with barriers to access, such as older adults.

Introduction

As information technology develops rapidly in the current era, telehealth is growing exponentially in use [ 1 - 3 ]. Particularly in the wake of the COVID-19 pandemic, the use of telehealth for both primary and specialist care has accelerated around the globe [ 4 ]. In particular, telehealth is being implemented at an increasing scale in various countries with aging populations to improve health care access and quality for growing numbers of patients with chronic diseases [ 5 - 9 ].

The internet hospital is 1 major emerging telehealth model that is distinct to China, a country with a particularly large aging population and a high chronic disease burden [ 10 - 12 ]. Designed to make health care services more available, convenient, affordable, and efficient, internet hospitals are a type of online platform through which certain health care services can be conducted remotely. There are 3 main types of internet hospitals—those initiated by physical hospitals, those jointly established by physical hospitals and business enterprises, and those initiated by business enterprises relying on physical medical institutions. Research suggests that internet hospitals initiated solely by physical hospitals are the most widespread type [ 13 ]. In terms of the target patient population, internet hospitals primarily aim to facilitate services for patients with common illnesses requiring relatively simple treatment [ 14 ], patients with chronic diseases (diabetes, hypertension, and cancer) [ 15 ], and patients in remote and rural areas [ 16 ].

However, the scope of internet hospitals goes beyond telemedicine services for patients. Services provided by internet hospitals can be classified into three categories, (1) “core medical services,” which mainly include follow-up care for in-person medical services, telemedicine consultations, guidance on chronic disease management, and guidance on medication use; (2) “non-core medical services,” which mainly include medical consultations between health care providers and remote education for health care providers; and (3) “convenience services,” which mainly include health care appointment scheduling, mobile payment for health services, remote examination of medical test results, and dispensation and distribution of some medications [ 10 , 13 , 17 , 18 ]. Thus, the internet hospital model has the potential to increase access to health care for patients and training for providers, and to decrease costs across the health care system.

A number of recent policies by the Chinese national government have promoted rapid development and uptake of the internet hospital model. In 2015 [ 19 ] and 2018 [ 20 ], the State Council issued guidelines promoting “‘Internet +’ Healthcare,” which emphasized the development of internet hospitals as part of the “Health China” strategy for health care reform. Concurrently in 2018, the National Health Commission formulated specific regulations on internet hospital management, which officially authorized internet hospitals to facilitate a range of telehealth services and marked the start of their standardized development [ 21 - 23 ]. In 2020, the National Health Commission issued the “Notice on Strengthening Informatization to Support the Prevention and Control of the Novel Coronavirus Pneumonia Epidemic,” emphasizing the advantages of internet hospitals in controlling the spread of the COVID-19 pandemic [ 24 ]. In the wake of these policies, by June 2023, the number of internet hospitals had reached more than 3000, and 364 million of China’s 1.079 billion internet users were using online medical services [ 25 , 26 ]. However, research suggests that most internet hospitals are not yet fully developed or providing the full scope of services intended to achieve these goals [ 27 - 29 ].

At this early stage of the model’s development, little research to date has evaluated the views of Chinese medical professionals and patients regarding internet hospitals. However, research on telehealth in other countries reveals that telehealth raises many new concerns and challenges alongside the aforementioned opportunities [ 30 - 32 ]. One of the most common concerns raised by patients is the potential for misdiagnosis due to the inability to conduct physical examinations through telehealth [ 33 , 34 ]. Particular groups such as lower-income older adults also commonly report barriers to the use of telehealth such as lack of familiarity with technology or limited access to technological devices or internet connections [ 35 ]. Smartphone data or internet connection problems can also lower patient satisfaction and limit access among rural patient populations [ 36 , 37 ]. Various groups of patients also commonly report feeling concerned about patient privacy and the security and protection of medical data when using telehealth [ 38 ].

In order to guide the direction of internet hospital development in China, further research is needed to examine the emerging challenges and opportunities to patient care presented by this country-specific telehealth platform. The aim of this study was to explore the experience and views of chronic disease physicians at 3 tertiary hospitals in Changsha, China, regarding opportunities and challenges presented by internet hospital care.

The methodology whereby this study was designed and conducted is reported following the items in the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist [ 39 ]. See Multimedia Appendix 1 [ 39 ] for more information.

Setting, Participant Recruitment, and Eligibility Criteria

We conducted in-depth, semistructured interviews with physicians at 3 tertiary hospitals in Changsha, Hunan Province, south central China. Inclusion criteria for participants were 18 years of age or older, experience working in internet hospitals, and employment in a chronic disease department at one of the study hospitals. Our rationale for these inclusion criteria was to select doctors who had work experience relevant to the research questions. Enrollment occurred over a 2-month period from April to May 2022. Using a purposive sampling approach, we obtained a list of doctors who had previous experience working in internet hospitals. We then messaged or called the doctors on the list to briefly explain the primary and secondary objectives of the study, invited them to share their perspectives related to the study, and asked them to be available for interviews. Out of the 28 doctors contacted, 26 agreed to participate, while 2 physicians declined due to a lack of time. Those who responded positively to the invitation were subsequently contacted by the author, YZ, either via WeChat (Tencent Holdings Limited) or telephone to schedule an interview.

We recruited participants until reaching data saturation, at which point no new information about the meaning of codes or themes and the relationship between them continued to appear [ 40 ].

In order to allow spontaneous answers and mitigate bias, participants were given minimum information in advance about specific interview topics.

Ethical Considerations

In April 2022, the research protocol was approved by the institutional review board of Xiangya Hospital, Central South University (#202204092). No prior relationships existed between study participants and members of the research team. Verbal informed consent was recorded via an audio recorder for each participant before participation. Participants were informed in advance that their interviews would be recorded, with the assurance that these recordings would be subject to encryption for security purposes, and they provided their verbal consent accordingly. All participants received a compensation of 200 RMB (1 CNY=US $0.15 on May 2022) for their participation, which was disbursed through a WeChat transfer. To protect the information of the interviewees, the interview data were deidentified in the process of transcription from audio recordings.

Data Collection

The interview guide was collaboratively developed and then subjected to pilot-testing by the research team. Throughout the concurrent phases of data collection and analysis, the interview guide underwent iterative refinement in response to emerging insights and participant responses. This adaptive approach is considered vital to the robustness of qualitative research [ 41 ]. Revisions were implemented subsequent to discussions involving YZ, JH, and Xiaomin Wang, aiming to clarify, define, and critically examine emerging content from interviews as relevant to the research questions. All questions from the finalized interview guide are listed in Textbox 1 .

Q1: What are your views on internet hospitals?

Q2: Could you tell me about your experience working in internet hospitals?

Q3: What do you think are the biggest advantages of internet hospitals? Can you give some examples?

Q4: What do you think are the most troubling or difficult aspects of internet hospitals? Can you give some examples?

Q5: How do you inform the patients who come to the internet hospitals before treatment?

Q6: Do you think there are any differences between doctor-patient communication in internet hospitals and physical hospitals?

Q7: What impact do you think internet hospitals have on doctor-patient communication? Can you give some examples?

Q8: What training have you participated in regarding internet hospitals? What do you think of this training?

Q9: What do you think about the current status of the development of internet hospital laws and regulations in China? How could they be improved? What other aspects can promote the further development of internet hospitals?

Q10: Is there anything else you would like to add about doctor-patient communication in internet hospitals?

Data for this study were collected from April to May 2022. We carried out interviews through a combination of online and offline modalities, depending on each participant’s preference and availability. Online interviews were conducted remotely by video call, via the mobile app WeChat. Offline interviews were conducted in private rooms at the study hospitals. The interviews were conducted in Mandarin Chinese by the authors, YZ (a postgraduate student) and Xiaomin Wang (an associate professor). Both interviewers have received professional training in qualitative interviewing and had extensive experience conducting qualitative research prior to this study.

The research team discussed possible probes and follow-up questions before beginning interviews, and interviewers used them when necessary to draw out more information relevant to the main research question. Concurrently, a second interviewer assumed the role of an observer to ensure the standardization of interview methods and to mitigate potential biases.

Interviews were audio recorded, transcribed, and uploaded into qualitative data management NVivo software (version 11; Lumivero) on password-protected computers to facilitate the analysis. Field notes made by interviewers during the interview process were also stored on password-protected computers, to be used for reference by the research team during analysis. Interviews ranged from 20 to 50 minutes long. Transcripts were sent to participants upon request, but no corrections, comments, or notes were made to transcripts.

Data Analysis

Analysis of the data was performed through conventional content analysis, using guidelines described by Hsieh and Shannon [ 42 ]. An advantage of conventional content analysis is that it avoids using preconceived categories, to generate codes inductively from the data. This modality is considered appropriate when current knowledge of the phenomenon being researched is limited [ 42 ].

Authors YZ and Xuxi Wang transcribed all interviews verbatim and reviewed transcripts several times to acquire a thorough understanding of the whole data set. They then read transcripts line-by-line and highlighted keywords and sentences from a set of initial transcripts, to generate primary codes that captured key concepts. Primary codes were repeatedly reviewed and revised through discussion among the authors and comparison across the transcripts. A finalized codebook including 17 codes and 61 subcodes was used to code all interviews, using NVivo software (version 11). Data saturation was reached after 26 interviews, once the research team determined by the consensus that we had interviewed a sufficiently varied sample of physicians from the 3 study hospitals, while also having obtained sufficiently content-rich data.

Following the coding of all transcripts, all coded segments of the interview data were translated into English by authors YZ and Xuxi Wang, native Mandarin speakers, and double-checked for accuracy by author JH, a native English speaker. Codes and subcodes were repeatedly reviewed and were grouped into clusters according to similarities and differences. Clusters of codes were then treated as subcategories and aggregated into the main categories that were representative of the key findings. These categories were repeatedly reviewed until fully developed, through a process of identifying and comparing exemplary excerpts for each code, category, and subcategory.

This process of analysis culminated in three main categories describing the experiences and views of chronic disease physicians regarding the opportunities and challenges presented by internet hospital care, (1) advancements and shortcomings in care access due to internet hospitals, (2) patients’ expectations versus limitations on doctors’ availability and the scope of services—implications for doctor-patient boundaries and trust, and (3) advantages and downsides of online communication for comprehension and informed consent. These main categories are shown in Textbox 2 , alongside the subcategories from which they were aggregated, and further explained in the results narrative below.

Advancements and shortcomings in care access due to internet hospitals

• Enhanced ability to conduct follow-up care for patients with chronic illness
• More efficient channels for health education
• Disparities in access (ie, for older adults, patients with lower education levels, patients with limited internet or technology access, rural patients)

Patients’ expectations versus limitations on doctors’ availability and the scope of services

• Patients’ expectations of doctors’ availability create unclear professional boundaries.
• Patients’ expectations of the service scope of internet hospitals affect doctor-patient trust.

Advantages and downsides of online communication for comprehension and informed consent:

• Doctors value having extra time to think carefully about replies to patients’ messages, compared to in-person communication.
• Internet hospitals’ restrictions on consultation times, procedures, and arbitrary rules or schedules can hinder effective patient communication.
• Doctors have concerns about the quality of online diagnoses and advice, as well as patient accuracy and comprehension, due to the limitations of online care.
• Doctors have concerns about the completeness and uniformity of clinical informed consent in internet hospitals.

Description of Study Participants

The 26 participants came from 3 different affiliated hospitals with 10, 5, and 11 participants interviewed from each hospital, respectively. Participants ranged in age from 29 to 49 years, and all 26 participants had PhD degrees. Only 5 participants stated that they had received specific training for working in internet hospitals, and 1 of them stated that training included discussion of clinical ethics in internet hospital care. We interviewed doctors from several departments involved in care for patients with chronic disease—oncology, cardiovasology, hematology, endocrinology, gastroenterology, nephrology, and infection departments. Aggregated participant characteristics are presented in Table 1 .

a Some percentages may not add up to 100 due to rounding.

Advancements and Shortcomings in Care Access Due to Internet Hospitals: Follow-Up Care, Health Education, and Disparities

Most doctors stated that internet hospitals affiliated with their physical hospitals of employment were still in the early stages of development and that their internet hospital work experience mainly took place in enterprise-initiated internet hospitals. Doctors stated that internet hospitals initiated by physical hospitals were “not fully operational yet,” (Dr B) and “the consultation volume of patients is relatively small” (Dr C). They also suggested that there was currently a “lack of incentives” (Dr D) to work in internet hospitals initiated by physical hospitals, whereas enterprise-initiated internet hospitals offered “higher income” (Dr B) and a setting where “doctors set their own prices” (Dr C).

In both internet hospitals initiated by physical hospitals and enterprise-initiated internet hospitals, doctors stated that the majority of their work consisted of online consultation for common or easily diagnosable diseases, and follow-up services for patients with chronic diseases who had previously received care at physical hospitals, such as adjusting medications and ordering medical tests to be scheduled in person. Most doctors were motivated to work for internet hospitals particularly because of the opportunity to be part of expanding follow-up care for patients with chronic diseases.

Most of the patients who come to the internet hospitals are chronic disease patients, with conditions such as hypertension, diabetes, coronary heart disease, etc. These patients have been clearly diagnosed in our hospital, and some of them need to be guided or communicated with about what needs to be paid attention to in the process of home-based management. For example, patients' blood pressure might fluctuate, or they can consult online if they have any uncomfortable symptoms, which is quite common. [Dr X]

Some doctors also believed based on experience that internet hospitals could serve as a more efficient channel to provide health education for patients, particularly for the management of chronic diseases.

We also feel that doctors in tertiary hospitals do not have much time to do health education with patients, but through the internet hospitals platforms, we can explain to patients the concept of health or a healthy way of life. For example, for a patient with heart failure, he isn’t expected to come back to the hospital again and again, because I have instilled him with an understanding of healthy lifestyle and diet, and the workload of the doctor will be reduced in the long run. [Dr C]

Despite the ways in which doctors felt internet hospitals expanded access to care and services, they had also observed disparities in access to internet hospitals across several groups, including older adults and patients with lower levels of education or technological literacy: “Older patients may not use smartphones or might need assistance to do so from family members” (Dr E).

Relatively speaking, if the patients come to the internet hospital for consultation, the education level of these patients will be higher, otherwise they will not be able to fully communicate with their doctor. [Dr O]

Most doctors mentioned that internet hospitals are especially suitable for patients with chronic diseases. Doctors also stated that while older adults are one of the most common groups of patients with chronic diseases, many older adults have difficulties in using or accessing internet hospitals (Dr D and Dr X). Some doctors mentioned that internet hospitals currently have limited connections and overlap with local health services in rural areas. They believed moving toward more connection with local services was an important goal—particularly because the demand at tertiary hospitals frequently outstrips resources (Dr P and Dr Q), and because many rural patients travel long distances to receive care at tertiary hospitals.

Even for follow-up visits for chronic diseases or common diseases, many patients will still go to tertiary hospitals. Instead, the patient can go to a qualified local hospital and send us the results of the test, and then [through internet hospital care,] we can advise the patient or tell him how to adjust the medication, or refer the patient to a tertiary hospital for testing. But at present, internet hospitals have not played a big enough role in these aspects. [Dr E]

Patients’ Expectations Versus Limitations on Doctors’ Availability and the Scope of Services: Implications for Doctor-Patient Boundaries and Trust

Patients’ expectations of doctors’ availability create unclear professional boundaries.

When asked about new challenges in patient care posed by internet hospitals, only 1 doctor mentioned risks related to patient privacy and data security.

The internet hospital platform where I am located requires patients to provide their name, gender, age, ID number, and other information, which can be seen on the doctor's portal, but as a doctor I will definitely not disclose the patient's private information but just give him diagnostic advice according to the necessary information provided by the patient. [Dr G]

By contrast, many doctors expressed concerns about their own privacy. Some doctors shared stories from their work in physical hospitals of willingly sharing their personal WeChat with patients in case patients had questions after discharge (Dr D, Dr M, and Dr R). While some doctors did not seem to mind-bending this boundary with patients, others remembered negative experiences when patients had sent messages making demands of doctors’ time at all hours (Dr E, Dr M, Dr P, and Dr Z). They also recalled times when patients obtained the doctor’s personal contact information through their own means and contacted them after leaving the hospital without the doctor’s consent (Dr V and Dr Z). As a result, some doctors had positive views of internet hospitals because they can serve as a means for online communication with patients without requiring the doctor to disclose their own personal contact information.

I prefer to use the official platform to communicate with patients, rather than through private WeChat or phone, because I really don't want to receive phone calls or text messages from patients after I work. But if the phone does ring, I will take into account that he is an old patient of mine, and I will still answer it, because I am not sure if there is any emergency. But for patients on such online hospital platforms, I rarely give them my phone number and personal WeChat. [Dr P]

Several doctors were also uncomfortable that they were required to post information about themselves when working on internet hospital platforms, such as their name, location, and credentials (Dr P and Dr X). Because the audience of patients in internet hospitals is wider, they worried that patients who were dissatisfied with care may have the ability to post negative information about them in public forums online, citing their personal information. As a result, doctors stated that they would be more cautious in diagnosis and giving advice when dealing with patients in internet hospitals with whom they were less familiar (Dr J, Dr O, and Dr V).

Patients’ Expectations of the Service Scope of Internet Hospitals Affect Doctor-Patient Trust

Another concern expressed by many doctors was that patients held unrealistic expectations of the scope of services that internet hospital doctors provide. Some doctors mentioned that some patients feel that just because they spend money in an internet hospital, they should be able to get all their problems solved at once or get immediate treatment (Dr E and Dr G)—when in reality in many cases, the doctor might need to conduct tests over multiple online consultations or might recommend that patients seek further medical services in offline, physical settings. Doctors were concerned that patients’ dissatisfaction with unmet expectations might generate distrust toward the doctor.

What patients don't know is that in fact, most of the time, seeing a doctor is a step-by-step process, and it is necessary to do examinations step-by-step to exclude diseases or diagnose diseases. They often have high expectations for the effect of consultation in internet hospitals, and they think that doctors should be able to diagnose their diseases at one time; and not only to diagnose them, but also to propose a treatment plan. [Dr E]

Some doctors suggested that this gap between expectations and reality could be especially strong for new patients whom the doctor had not seen previously for in-person care. They described that they often recommend for new internet hospital patients to go to physical hospitals to be examined before receiving further internet hospital care or advice, and that patients who are expecting immediate solutions can find this disappointing (Dr U and Dr X). One doctor suggested the need to educate patients and the general public on the scope of internet hospital care, in light of this mismatch in patients’ and doctors’ expectations (Dr Y).

However, some doctors raised concerns about issues related to doctor-patient trust that had less to do with adjusting patients’ expectations and more to do with the format of online communication itself.

Doctor-patient trust will be better in physical hospitals. Because the doctor-patient relationship is a very special relationship, offline communication can observe the patient's expression, speed of speech, action, etc, and is more suitable for empathy with patients. When it comes to the doctor-patient relationship and trust, I think face-to-face consultation is still necessary. [Dr J]

Face-to-face communication in physical hospitals may be more detailed, because if it is through text messages or phone calls, I may be able to talk to the patient in a few words, but if the patient is communicating in our hospital, it may take me half an hour. Because there is unequal information in medicine itself, the patient himself is not very clear about medicine, and without adequate communication, there is no trust between doctors and patients. [Dr W]

Advantages and Downsides of Online Communication for Comprehension and Informed Consent

Doctors working in internet hospitals mainly used pictures and texts, and rarely video calls, to communicate with patients. Some doctors valued the extra time gained by this format to think carefully about their replies to patients’ messages (Dr E, Dr U, and Dr X). However, most doctors pointed out how the lack of nonverbal communication could increase miscommunication and misunderstandings.

Online communication is through typing, and some doctors can't see the facial expressions of patients, which is very inconvenient. The communication between doctors and patients may need body language, facial expressions and other aspects.... I want the patient to really understand me in terms of attitude or tone or feeling or whatever. [Dr G]

Doctors also expressed dissatisfaction with limitations on the time and procedures for consultation through various internet hospitals, and how sometimes arbitrary rules or schedules hindered communication with patients.

The doctors in our department need to be on duty every month in the internet hospitals. When it is my turn to be on duty, a patient will send his questions to me through the platform, but I think this mode is not good. For example, the patient might leave a message for me, but I am busy and don’t reply to him in time, and he may not see my reply in time when I reply. If I go back and forth with him several times, this problem will not be solved until I come back on duty next month, and then the patient's problem will not be solved at all. [Dr C]

The internet hospital at our hospital stipulates that patients can ask five questions at a time.... Sometimes doctors are not able to inquire in detail in order to understand the condition. [Dr G]

Doctors had concerns about the quality of diagnoses and advice that they provided online, due to the inability to do direct physical examination. These concerns were intensified by their perception that many patients could not describe their symptoms clearly and accurately.

Currently, a lot of people still lack of basic medical knowledge, which will lead to ineffective or inefficient consultation on the internet, because they cannot describe their own symptoms, or cannot collect their own data and then summarize it. Patients cannot provide information about their condition sufficiently and accurately, which will seriously affect the efficiency of consultation. [Dr E]

Two doctors also specifically mentioned the difficulty in internet hospital care of not being able to use the “four-diagnosis method”—a method used by doctors in traditional Chinese medicine for diagnosing illness, including diagnosis through observation, diagnosis through auscultation and olfaction, diagnosis through inquiry, and diagnosis through pulse feeling [ 43 ]. Although doctors in this study practiced mainly “Western” medicine, they described integrating certain traditional practices such as this method into their care at physical hospitals (Dr D and Dr H).

In light of concerns about the potential for miscommunication with patients, a few doctors also expressed uncertainty about the completeness and uniformity of clinical informed consent as it is currently practiced in internet hospitals. While they believed that a standardized process of online informed consent for medical advice and treatment was needed, they did not know of any relevant laws or procedures.

Because we have so little time to communicate online, and such a narrow scope of care services, we don't usually obtain informed consent online. I might listen to the patient explain his symptoms. I might tell him what tests he needs before I give my advice, or if I'm dealing with a familiar patient, I might just prescribe his medication, so there's no need for informed consent. However, I think how to issue online informed consent, whether online informed consent is legally effective, and how to sign online informed consent all need to be considered. This is also for the protection of medical staff. [Dr U]

Principal Findings

This study sheds light on previously underresearched aspects of internet hospitals in China, as both the first interview study to examine physicians’ perceptions of internet hospitals and one of the few studies on internet hospitals conducted in China outside of its most major cities. Our research revealed that physicians see enhanced opportunities in internet hospitals to conduct follow-up care for patients with chronic illnesses and to provide health education. However, physicians noted disparities in access for different groups, such as older adults, patients with lower education levels, patients with limited internet or technology access, and rural patients. One particularly novel finding was the conflict between patients’ expectations and the reality of limitations on doctors’ availability and the scope of services available through internet hospitals. Physicians perceived that this gap affected both boundaries and trust in the doctor-patient relationship. Physicians also discussed opportunities and challenges in doctor-patient communication, including issues of comprehension and informed consent. Considering that the development of internet hospitals involves multiple industries, including medical institutions, national policymaking departments, and technology providers, we raise several suggestions below on physician training, patient education, regulations, and design, as well as directions for future research.

Training for Doctors

Internet hospital care involves real-time online sharing of medical data. Information about both doctors and patients is centralized and easily accessible to authorized users on the internet hospital platform. Some doctors in our study were uncomfortable when required to publicly post their names, basic personal information, and credentials on internet hospital platforms because it might make them more vulnerable to public criticism. Doctors’ reasons for being concerned about this were in line with previous research showing a high degree of conflict in the doctor-patient relationship in China [ 44 , 45 ]. This underscores the importance of current efforts both locally and internationally that aim to rebuild trust in the doctor-patient relationship [ 46 - 48 ]. Considering doctors’ concerns about patients requesting for them to disclose their WeChat in both physical and internet hospital work, communication skills training for doctors should prepare doctors for how to interact with patients with empathy and care, while also maintaining their preferred professional boundaries.

It was also notable that only 1 doctor who was interviewed discussed concerns related to patient privacy and data security when asked about challenges presented by internet hospitals. By contrast, Li et al’s [ 49 ] study on the determinants of patients’ use of internet hospitals in China showed that while patients generally desire to use internet hospitals, they are apprehensive about the associated risk of their personal information being leaked. Due to the heightened potential for data leaks and breaches of patient health information associated with the use of internet hospitals, it is imperative that health care professionals undergo training to raise their awareness of data security precautions. For instance, physicians should be trained on proactive measures that they can take to guarantee that the internet hospital services they are affiliated with implement adequate security protocols around patient information. Furthermore, physicians should be trained to communicate with internet hospital patients or their legal proxies about potential risks related to data security and to apprise them of protective measures enacted to safeguard information. Future research should also evaluate the frequency with which data leaks and breaches in internet hospitals actually occur.

Findings from our study also suggest that internet hospitals have led to changes in doctor-patient communication. Doctors in our study considered it to be an advantage of internet hospital care that they generally had more time to communicate with patients compared to in-person care. However, a previous study on internet hospitals suggested that while doctors can obtain key information from patients within a few minutes through in-person communication and examination, information received in the same amount of time online tends to be more limited [ 43 ]. Research conducted by Deng et al [ 50 ] also highlights that engaging in online consultation work while simultaneously engaging in a main career providing in-person medical consultation may place excessive demands on doctors’ time and energy. This phenomenon of work overload could potentially impede the widespread adoption of internet hospitals and introduce added risk to medical practice.

Relatedly, doctors in our study mentioned that when working in internet hospitals, they could only communicate with patients in the form of text, pictures, or video-based consultations, and that they had to rely largely on patient self-report. Both of these factors caused doctors to worry about the accuracy of their diagnoses. This aligns with recent research showing that about 70% of surveyed health care providers believe communication difficulties between patients and health care providers result in online consultations being insufficient [ 51 ], and about 70% of providers report feeling apprehensive about the possibility of misdiagnosis when providing care through internet hospitals [ 51 ]. Recent research has also found that patients who use internet hospitals have more negative views on the doctor-patient relationship than nonusers—including both interpersonal factors such as the degree to which patients trust doctors and practical factors such as the degree to which patients agree with their doctors’ medical opinions. Studies from other countries have similarly shown that telehealth can present new challenges or deficiencies in communication [ 52 - 55 ].

To address such challenges, telehealth communication competencies need to become a core component of both future research and physician training for internet hospitals in China—just as similar competencies are emerging as a priority for telehealth enhancement around the world [ 56 ]. Physicians providing care through internet hospitals should undergo standardized training for web-based communication skills, as research from other countries suggests such training can adapt interpersonal skills to the telehealth environment [ 57 ] and enhance empathic expression. More training for physicians on this skill set might reduce their apprehension about communicating through internet hospitals and assist them in communicating in a manner that improves outcomes for patients. Considering that doctors in our study expressed concern about patient comprehension and diagnostic accuracy, further research is also needed to evaluate and establish methods for measuring patient satisfaction, patient comprehension of information communicated by doctors, and diagnostic accuracy in internet hospital care. Future research should also examine the feasibility of integrating traditional Chinese medical practices such as the four-diagnosis method into telehealth care in China.

Education for Patients or the Public

Findings from our study highlight new challenges in the doctor-patient relationship posed by internet hospital care. One especially novel finding in our study was doctors’ perception that patients subconsciously expected them to be online 24 hours a day, while doctors actually had limited hours working in the internet hospital and could not meet this expectation. Particularly when patients still needed to ask questions after the end of the physician’s available time for consultation, doctors described the risk of conflicts with patients. These findings suggest many patients may be unaware when message-based interactions with physicians in internet hospitals are discontinuous or asynchronous. Therefore, public information about internet hospitals should specify the boundaries of physicians’ availability for internet hospital consultations. While the scope of services may expand as internet hospitals continue to develop, information disseminated to the public should make it clear that internet hospital care is currently only intended for either follow-up care for previously diagnosed patients with chronic diseases, or for new patients with common and more easily diagnosable conditions. Finally, public education should equip patients or their proxies for distinct ways in which they might self-advocate for optimal care in the context of internet hospitals compared to in-person care. This may involve the development of interventions such as question-prompt lists that are specific to equipping patients for internet hospital consultations.

Regulations and Laws

Doctors in our study believed that difficulties with nonverbal communication in internet hospitals often led to miscommunication and misunderstanding, and many raised concerns that there were no specific laws regulating online doctor-patient communication. As a result, most doctors in our study expressed that they felt they were walking on eggshells concerning possible conflicts with patients. This builds on findings from the “2022 China E-hospital development research report” [ 51 ], in which one of the most common suggestions made by health care providers for the further development of internet hospitals was to standardize legal protection for doctors practicing in internet hospitals. Gaps in relevant laws and regulations may reduce the willingness of risk-conscious clinicians to provide medical services through internet hospitals.

Existing internet hospital laws and regulations in China are still mainly in the trial stage [ 51 , 58 ], and are being outpaced by evolving challenges in internet hospital care. The doctors in our study believed that internet hospitals may increase the difficulty of diagnosis and treatment, increase medical safety risks related to miscommunication, and increase the risk of medical malpractice liability. Research by Zhi et al describes how the inability of doctors to perform physical examinations or certain laboratory or imaging examinations through internet hospitals may compromise the accuracy of doctors’ judgments [ 59 ]. However, the legal responsibilities of physical medical institutions, internet hospitals, and doctors regarding issues such as these have not been fully clarified. We suggest that further refinement and clarification of these and other aspects of internet hospital law will help doctors feel more protected in their work and increase the motivation of doctors to work in internet hospitals.

Doctors in our study mentioned that China also lacks detailed legal provisions on the implementation of online informed consent. Internationally recognized ethics standards highlight 4 core elements of informed consent—capacity to consent, information disclosure, comprehension, and voluntary authorization [ 60 ]. Informed consent issues involved in telehealth in other countries are similar to those described by doctors working in internet hospitals in this study, namely, the degree of discernment required from providers to ensure that patients are sufficiently informed to provide consent increases dramatically in telehealth [ 61 ]. In the United States, different states have different regulations on remote informed consent, and no federal policy has been formed at present. Some states require patients to fill out and sign written consent forms, while others do not [ 62 ]. In China, the Administrative Measures for internet hospitals stipulate that “internet hospitals must warn patients of risks and obtain informed consent from patients” [ 22 ]. However, current laws in China do not provide clear rules regarding the validity of electronic signatures for informed consent in internet hospital care. Informed consent in internet hospital care also involves unique information security issues due to the use of electronic health records, but there is currently no specific legal guidance for internet hospital platform developers or doctors concerning data security protection of informed consent in internet hospital care.

Tackling Disparities

Our research revealed that while older adults are at higher risk for chronic illness and are the main target population for internet hospitals, they are also reported by doctors to experience a number of barriers to internet hospital use. This finding aligns with research from various countries showing that older adults are less likely than younger patients to express positive attitudes toward using telehealth [ 63 , 64 ]. Health care providers in China and other countries have observed that older adults may be apprehensive about telehealth due to difficulty in operating computers or smartphones [ 65 ], may need the help of care partners to log into telehealth accounts successfully, and may need more time on average to download and set up applications [ 43 , 66 ]. Previous surveys have also shown that medical personnel believe that the low efficiency of online communication between doctors and patients and the low internet use rate of some patient groups (such as older adults) are the main factors hindering the development of internet hospitals [ 51 ].

Previous research in various countries has shown that the ease of use and perceived usefulness of telehealth systems have a positive impact on the acceptance of telehealth in patients who are older [ 67 , 68 ]. However, to date, China has not established an effective quality control system for internet hospitals [ 27 , 69 ], and the aforementioned ways in which internet hospitals currently pose increased risks for patient safety may affect general patients’, let alone older adults’ willingness to use them [ 49 ]. We recommend that the needs of older adults be considered in the design and development of internet hospital platforms and that older adults participate in the system design process [ 70 , 71 ]. Community health workers may be a workforce that could be mobilized to support telehealth training efforts among patients who are older, assist individuals with limited telehealth literacy in attending online appointments, and provide culturally and linguistically appropriate information about telehealth to rural patients and communities [ 64 , 72 , 73 ]. In general, health care organizations should invest in developing internet hospitals that are functional and easy to use. Drawing from research on telehealth design improvement in other countries, internet hospitals could be designed with features in mind to help physicians communicate more clearly with patients, such as providing notifications to physicians when patients read messages [ 74 ]. In addition, it may be beneficial for platforms to provide training materials to patients when patients register and log into internet hospitals for the first time [ 75 ]. Considering that a major goal of internet hospital development is to expand health care access, it will be crucial to address disparities in internet hospital use through these and other educational and design considerations.

Limitations

This study should be interpreted in the light of certain limitations. As most participants interviewed were attending physicians, findings may not be generalizable to the perspective of other health care workers or patients. The generalizability of our study findings from a single region and time point may also be limited, as there may be variations in internet hospital features and practice in other regions in China, and over time as internet hospitals continue to develop rapidly.

Conclusions

This study explored the experience and views of physicians in 3 tertiary hospitals in Changsha, China regarding access to care, patients’ expectations versus the reality of services, and doctor-patient communication in internet hospital care. Findings from this study indicate that there is a need to train physicians in telehealth-specific communication skills. National policymaking departments should also further refine laws and regulations concerning internet hospitals, particularly those related to online informed consent. Technology developers should take the needs of older adults into particular account in the design of internet hospital platforms.

Acknowledgments

This study was supported by Major Program of National Social Science Fund of China, Research on Moral Issues in the Field of Contemporary Science and Technology (22&ZD044), the Xiangya Medical Humanities Series: Principles of Biomedical Ethics (monograph Award) Fund (KTZZPT019), Hunan Provincial Innovation Foundation for Postgraduate (CX20220133), the Fundamental Research Funds for the Central Universities of Central South University (2022ZZTS0041), the China Scholarship Council (CSC,202206370069), and the Natural Science of Changsha City (kq2202362).

Data Availability

The data generated and analyzed during this study are available from the corresponding author upon reasonable request.

Authors' Contributions

YZ and XL conceptualized this study and designed the methodology. YZ and Xiaomin Wang conducted the interviews for data collection. Xuxi Wang and YZ transcribed the interviews. YZ, Xuxi Wang, YW, and XZ conducted and provided resources for preliminary analysis of the data. YZ and JH wrote and edited the paper. JH, XL, and XZ oversaw the implementation of all study activities. All authors read and approved the final paper.

Conflicts of Interest

None declared.

COREQ (Consolidated Criteria for Reporting Qualitative Research) checklist.

• Barbosa W, Zhou K, Waddell E, Myers T, Dorsey ER. Improving access to care: telemedicine across medical domains. Annu Rev Public Health. Apr 01, 2021;42:463-481. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kirkland EB, DuBose-Morris R, Duckett A. Telehealth for the internal medicine resident: a 3-year longitudinal curriculum. J Telemed Telecare. Oct 2021;27(9):599-605. [ CrossRef ] [ Medline ]
• Losorelli SD, Vendra V, Hildrew DM, Woodson EA, Brenner MJ, Sirjani DB. The future of telemedicine: revolutionizing health care or flash in the pan? Otolaryngol Head Neck Surg. Aug 2021;165(2):239-243. [ CrossRef ] [ Medline ]
• Hafner M, Yerushalmi E, Dufresne E, Gkousis E. The potential socio-economic impact of telemedicine in Canada. Rand Health Q. Jun 2022;9(3):6. [ FREE Full text ] [ Medline ]
• Alhamam NM, Buhalim RA, Almakhayitah IH, AlBahr AW, AlYaeesh IA. Telemedicine for musculoskeletal care during the COVID-19 pandemic: evaluating readiness of Saudi citizens. Cureus. Feb 16, 2021;13(2):e13380. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Bhaskar S, Bradley S, Chattu VK, Adisesh A, Nurtazina A, Kyrykbayeva S, et al. Telemedicine across the globe-position paper from the COVID-19 pandemic health system resilience PROGRAM (REPROGRAM) international consortium (part 1). Front Public Health. 2020;8:556720. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cioti AM, Stanescu AMA, Grajdeanu IV, Serban B, Popescu E, Bratu OG, et al. Telemedicine in Europe—current status and future perspectives. Rev Med Mod. 2019;26(4):165-168. [ FREE Full text ] [ CrossRef ]
• Raja M, Bjerkan J, Kymre IG, Galvin KT, Uhrenfeldt L. Telehealth and digital developments in society that persons 75 years and older in European countries have been part of: a scoping review. BMC Health Serv Res. Oct 26, 2021;21(1):1157. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Brucksch S. Japan and its rapid ageing society: does e-health technology provide a solution? J Aging Sci. 2018;6(2):1-4. [ FREE Full text ] [ CrossRef ]
• Han Y, Lie RK, Guo R. The internet hospital as a telehealth model in China: systematic search and content analysis. J Med Internet Res. Jul 29, 2020;22(7):e17995. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Scott R, Mars M. Telehealth in the developing world: current status and future prospects. SHTT. Feb 2015;2015(3):25-37. [ FREE Full text ] [ CrossRef ]
• Yip W, Fu H, Chen AT, Zhai T, Jian W, Xu R, et al. 10 years of health-care reform in China: progress and gaps in Universal Health Coverage. Lancet. Sep 28, 2019;394(10204):1192-1204. [ CrossRef ] [ Medline ]
• 2021 China E-hospital development report. National Telemedicine and Connected Health Center and CN-Healthcare. 2021. URL: https://zk.cn-healthcare.com/doc-show-53644.html [accessed 2023-09-17]
• Chen X, Wu X, Zhang Q, Jing R, Cheng W, Tian J, et al. The construction and operational models of internet hospitals in China: a hospital-based survey study. BMC Health Serv Res. Jun 21, 2023;23(1):669. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Houston TK, Allison JJ. Users of Internet health information: differences by health status. J Med Internet Res. 2002;4(2):E7. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lai Y, Chen S, Li M, Ung COL, Hu H. Policy interventions, development trends, and service innovations of internet hospitals in China: documentary analysis and qualitative interview study. J Med Internet Res. Jul 20, 2021;23(7):e22330. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yu T, Jin C, Wu X, Yue D. Implementation of shared decision-making within internet hospitals in China based on patients' needs: feasibility study and content analysis. JMIR Form Res. Jan 06, 2023;7:e39965. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tu J, Wang C, Wu S. The internet hospital: an emerging innovation in China. Lancet Glob Health. Aug 2015;3(8):e445-e446. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Guiding opinions of the State Council on actively promoting the "Internet +" action. The State Council of the People's Republic of China. 2015. URL: https://www.gov.cn/zhengce/content/2015-07/04/content_10002.htm [accessed 2023-09-16]
• Opinions on promoting the development of "Internet + medical health". General Office of the State Council. 2018. URL: https://www.gov.cn/zhengce/content/2018-04/28/content_5286645.htm [accessed 2023-09-16]
• Internet diagnosis and treatment management measures (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Internet hospital management measures (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Telemedicine service management standards (trial implementation). National Health Commission of the People's Republic of China. 2018. URL: https://www.gov.cn/gongbao/content/2019/content_5358684.htm [accessed 2023-09-16]
• Notice of the general office of the NHC on strengthening informatization to support the prevention and control of the novel coronavirus pneumonia epidemic. National Health Commission of China. 2020. URL: https://www.gov.cn/zhengce/zhengceku/2020-02/05/content_5474692.htm [accessed 2023-09-16]
• The 52nd statistical report on China's internet development. China Internet Network Information Center. 2023. URL: https://www.cnnic.cn/NMediaFile/2023/0908/MAIN1694151810549M3LV0UWOAV.pdf [accessed 2024-02-28]
• There are more than 3,000 internet hospitals, and it is necessary to explore a better development path. Health Times. 2023. URL: http://www.jksb.com.cn/index.php?m=content&c=index&a=show&catid=788&id=200853 [accessed 2024-02-28]
• Qiu Y, Liu Y, Ren W, Qiu Y, Ren J. Internet-based and mobile-based general practice: cross-sectional survey. J Med Internet Res. Sep 25, 2018;20(9):e266. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Xie X, Zhou W, Lin L, Fan S, Lin F, Wang L, et al. Internet hospitals in China: cross-sectional survey. J Med Internet Res. Jul 04, 2017;19(7):e239. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Yuan J, He F. Status quo and problems of the internet hospitals in China. Chin Rural Health Serv Adm. 2023;43(02):126-128+140. [ CrossRef ]
• Pogorzelska K, Chlabicz S. Patient satisfaction with telemedicine during the COVID-19 pandemic-a systematic review. Int J Environ Res Public Health. 2022;19(10):6113. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sagaro GG, Battineni G, Amenta F. Barriers to sustainable telemedicine implementation in Ethiopia: a systematic review. Telemed Rep. 2020;1(1):8-15. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Şahin E, Yavuz Veizi BG, Naharci MI. Telemedicine interventions for older adults: a systematic review. J Telemed Telecare. Feb 2024;30(2):305-319. [ CrossRef ] [ Medline ]
• Imlach F, McKinlay E, Middleton L, Kennedy J, Pledger M, Russell L, et al. Telehealth consultations in general practice during a pandemic lockdown: survey and interviews on patient experiences and preferences. BMC Fam Pract. Dec 13, 2020;21(1):269. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kumar S, Kumar A, Kumar M, Kumar A, Arora R, Sehrawat R. Feasibility of telemedicine in maintaining follow-up of orthopaedic patients and their satisfaction: a preliminary study. J Clin Orthop Trauma. Oct 2020;11(Suppl 5):S704-S710. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Latulipe C, Gatto A, Nguyen HT, Miller DP, Quandt SA, Bertoni AG, et al. Design considerations for patient portal adoption by low-income, older adults. Proc SIGCHI Conf Hum Factor Comput Syst. Apr 2015;2015:3859-3868. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Greenberg AJ, Haney D, Blake KD, Moser RP, Hesse BW. Differences in access to and use of electronic personal health information between rural and urban residents in the United States. J Rural Health. 2018;34(Suppl 1):s30-s38. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Johnson BA, Lindgren BR, Blaes AH, Parsons HM, LaRocca CJ, Farah R, et al. The new normal? Patient satisfaction and usability of telemedicine in breast cancer care. Ann Surg Oncol. 2021;28(10):5668-5676. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Solimini R, Busardò FP, Gibelli F, Sirignano A, Ricci G. Ethical and legal challenges of telemedicine in the era of the COVID-19 pandemic. Medicina (Kaunas). 2021;57(12):1314. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tong A, Sainsbury P, Craig J. Consolidated Criteria for Reporting Qualitative Research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349-357. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591-608. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Klem NR, Bunzli S, Smith A, Shields N. Demystifying qualitative research for musculoskeletal practitioners part 5: rigor in qualitative research. J Orthop Sports Phys Ther. 2022;52(2):60-62. [ CrossRef ] [ Medline ]
• Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277-1288. [ CrossRef ] [ Medline ]
• Li Y, Hu H, Rozanova L, Fabre G. COVID-19 and internet hospital development in China. Epidemiologia (Basel). 2022;3(2):269-284. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Jiang MM, Wu ZY, Tu AX. Research on the cooperative governance path of multiple stakeholders in doctor-patient disputes under the environment of information asymmetry. Int J Environ Res Public Health. 2023;20(2):1597. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhao D, Zhang Z. Changes in public trust in physicians: empirical evidence from China. Front Med. 2019;13(4):504-510. [ CrossRef ] [ Medline ]
• Du L, Xu J, Chen X, Zhu X, Zhang Y, Wu R, et al. Rebuild doctor-patient trust in medical service delivery in China. Sci Rep. 2020;10(1):21956. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fu Y, Tang T, Long J, Lin B, Li J, Quan G, et al. Factors associated with using the internet for medical information based on the doctor-patient trust model: a cross-sectional study. BMC Health Serv Res. 2021;21(1):1268. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tucker JD, Wong B, Nie JB, Kleinman A, Patient-Physician Trust Team. Rebuilding patient-physician trust in China. Lancet. 2016;388(10046):755. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Li D, Hu Y, Pfaff H, Wang L, Deng L, Lu C, et al. Determinants of patients' intention to use the online inquiry services provided by internet hospitals: empirical evidence from China. J Med Internet Res. 2020;22(10):e22716. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Deng W, Yang T, Deng J, Liu R, Sun X, Li G, et al. Investigating factors influencing medical practitioners' resistance to and adoption of internet hospitals in China: mixed methods study. J Med Internet Res. 2023;25:e46621. [ FREE Full text ] [ CrossRef ] [ Medline ]
• 2022 China E-hospital development research report. Shanghai Jiao Tong University. 2022. URL: https://www.100ec.cn/detail--6620464.html [accessed 2023-01-08]
• Alpert JM, Dyer KE, Lafata JE. Patient-centered communication in digital medical encounters. Patient Educ Couns. 2017;100(10):1852-1858. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Alpert JM, Krist AH, Aycock RA, Kreps GL. Applying multiple methods to comprehensively evaluate a patient portal's effectiveness to convey information to patients. J Med Internet Res. May 17, 2016;18(5):e112. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Gerber DE, Beg MS, Duncan T, Gill M, Craddock Lee SJ. Oncology nursing perceptions of patient electronic portal use: a qualitative analysis. Oncol Nurs Forum. Mar 01, 2017;44(2):165-170. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Sieck CJ, Hefner JL, Schnierle J, Florian H, Agarwal A, Rundell K, et al. The rules of engagement: perspectives on secure messaging from experienced ambulatory patient portal users. JMIR Med Inform. 2017;5(3):e13. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van Galen LS, Wang CJ, Nanayakkara PWB, Paranjape K, Kramer MHH, Car J. Telehealth requires expansion of physicians' communication competencies training. Med Teach. 2019;41(6):714-715. [ CrossRef ] [ Medline ]
• Wright HH, O'Shea MC, Sekula J, Mitchell LJ. Assessment of communication skills using telehealth: considerations for educators. Front Med (Lausanne). 2022;9:841309. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huang M, Wang J, Nicholas S, Maitland E, Guo Z. Development, status quo, and challenges to China's health informatization during COVID-19: evaluation and recommendations. J Med Internet Res. 2021;23(6):e27345. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Zhi L, Yin P, Ren J, Wei G, Zhou J, Wu J, et al. Running an internet hospital in China: perspective based on a case study. J Med Internet Res. 2021;23(9):e18307. [ FREE Full text ] [ CrossRef ] [ Medline ]
• World Medical Association. World Medical Association declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Blake JH, Schwemmer MK, Sade RM. The patient-surgeon relationship in the cyber era: communication and information. Thorac Surg Clin. 2012;22(4):531-538. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Fields BG. Regulatory, legal, and ethical considerations of telemedicine. Sleep Med Clin. 2020;15(3):409-416. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Chiang KF, Wang HH, Chien IK, Liou JK, Hung CL, Huang CM, et al. Healthcare providers' perceptions of barriers in implementing of home telecare in Taiwan: a qualitative study. Int J Med Inform. 2015;84(4):277-287. [ CrossRef ] [ Medline ]
• Gray DM, Joseph JJ, Olayiwola JN. Strategies for digital care of vulnerable patients in a COVID-19 world-keeping in touch. JAMA Health Forum. 2020;1(6):e200734. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Miller DP, Latulipe C, Melius KA, Quandt SA, Arcury TA. Primary care providers' views of patient portals: interview study of perceived benefits and consequences. J Med Internet Res. 2016;18(1):e8. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Varsi C, Gammon D, Wibe T, Ruland CM. Patients' reported reasons for non-use of an internet-based patient-provider communication service: qualitative interview study. J Med Internet Res. 2013;15(11):e246. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cimperman M, Brenčič MM, Trkman P. Analyzing older users' home telehealth services acceptance behavior-applying an extended UTAUT model. Int J Med Inform. 2016;90:22-31. [ CrossRef ] [ Medline ]
• Zhou M, Zhao L, Kong N, Campy KS, Qu S, Wang S. Factors influencing behavior intentions to telehealth by Chinese elderly: an extended TAM model. Int J Med Inform. 2019;126:118-127. [ CrossRef ] [ Medline ]
• 2020 research report on the development of internet hospitals in China. Institute of Healthcare. 2020. URL: http://zk.cn-healthcare.com/doc-show-39773.html [accessed 2023-09-17]
• Bhattarai P, Phillips JL. The role of digital health technologies in management of pain in older people: an integrative review. Arch Gerontol Geriatr. 2017;68:14-24. [ CrossRef ] [ Medline ]
• Cosco TD, Fortuna K, Wister A, Riadi I, Wagner K, Sixsmith A. COVID-19, social isolation, and mental health among older adults: a digital catch-22. J Med Internet Res. 2021;23(5):e21864. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Huang W, Long H, Li J, Tao S, Zheng P, Tang S, et al. Delivery of public health services by Community Health Workers (CHWs) in primary health care settings in China: a systematic review (1996-2016). Glob Health Res Policy. 2018;3:18. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Lohr AM, Ingram M, Nuñez AV, Reinschmidt KM, Carvajal SC. Community-clinical linkages with community health workers in the United States: a scoping review. Health Promot Pract. 2018;19(3):349-360. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Laukka E, Huhtakangas M, Heponiemi T, Kujala S, Kaihlanen AM, Gluschkoff K, et al. Health care professionals' experiences of patient-professional communication over patient portals: systematic review of qualitative studies. J Med Internet Res. 2020;22(12):e21623. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Hefner JL, Sieck CJ, Walker DM. Patient and physician perspectives on training to improve communication through secure messaging: clarifying the rules of engagement. Health Care Manage Rev. 2022;47(1):3-11. [ CrossRef ] [ Medline ]

Abbreviations

Edited by T de Azevedo Cardoso; submitted 22.03.23; peer-reviewed by Y Cao, N Mungoli, A Gangadhara Rao; comments to author 06.09.23; revised version received 27.09.23; accepted 26.02.24; published 29.03.24.

©Yuqiong Zhong, Jessica Hahne, Xiaomin Wang, Xuxi Wang, Ying Wu, Xin Zhang, Xing Liu. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 29.03.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Explainer-Why Did the Baltimore Bridge Collapse and What Is the Death Toll?

National Transportation Safety Board (NTSB) investigators work on the cargo vessel Dali, which struck and collapsed the Francis Scott Key Bridge, in Baltimore, Maryland, U.S. March 27, 2024. Peter Knudson/NTSB/Handout via REUTERS

By Lisa Shumaker

(Reuters) -The biggest operational crane on the U.S. Eastern Seaboard will begin clearing the wreckage of Baltimore's Francis Scott Key Bridge days after a cargo ship crashed into it, sending the span crashing into the harbor and killing six construction workers.

Replacing the bridge will likely take years, but the port could reopen within weeks if debris is rapidly removed, according to a Moody's report.    

WHAT IS THE DEATH TOLL SO FAR?

The six victims of the bridge collapse were all immigrants from Mexico and Central America who were fixing potholes on the bridge.

Divers recovered the bodies of two men on Wednesday. They were identified as Alejandro Hernandez Fuentes, 35, of Baltimore, originally from Mexico, and Dorlian Castillo, 26, of nearby Dundalk, originally from Guatemala.

Authorities are still trying to recover the bodies of the other four men in the 50-foot-deep (15 meter) waters surrounding the twisted ruins due to treacherous conditions. They are Maynor Suazo from Honduras; Jose Lopez from Guatemala; Miguel Luna from El Salvador; and another whose name has not been released. Two other workers were rescued.

Authorities saved lives by stopping vehicles from using the bridge after the ship sent out a mayday call. 

The ship also dropped its anchors to slow down, buying time to clear the bridge. 

WHEN DID THE BALTIMORE BRIDGE COLLAPSE?

Shortly after 1 a.m. EDT (0500 GMT) on Tuesday, a container ship named the Dali was traveling down the Patapsco River on its way to Sri Lanka. At 1:24 a.m., it suffered a total power failure and all its lights went out. 

Three minutes later, at 1:27 a.m., the container ship struck a pylon of the bridge, crumpling almost the entire structure into the water. 

Less than a minute before impact, a first responder on emergency radio responded to the crew's mayday call by sending officers to halt traffic onto the bridge.

Without their fast work, the scale of the disaster may have been far greater, even during the early morning hours when vehicular traffic is relatively light.

Tuesday's disaster may be the worst U.S. bridge collapse since 2007, when a design error caused the I-35W bridge in Minneapolis to plunge into the Mississippi River, killing 13 people.

WHY DID THE BRIDGE COLLAPSE?

Bridges such as the one in Baltimore are classified as "fracture critical" by the federal government - meaning that if one portion of the bridge collapses, the rest of the structure falls. There are more than 16,800 such spans in the U.S., according to the Federal Highway Administration.

The head of the National Transportation Safety Board said the bridge lacked structural engineering redundancies common to newer spans, making it more vulnerable to catastrophic collapse.

The Key Bridge opened in 1977 - three years before a similar vessel collision of the Sunshine Skyway Bridge in Tampa Bay, Florida, killed 35 people, and prompted bridge designers to implement better protections for foundation piers.

WHO WILL PAY FOR THE DAMAGE AND HOW MUCH WILL THE BRIDGE COST?

President Joe Biden promised to visit Baltimore soon and said he wanted the federal government to pay to rebuild the bridge. The Transportation Department on Thursday awarded $60 million in "quick release" emergency relief funds to aid in clearing debris and begin the process of rebuilding. To replace the bridge, Congress would need to approve funding. After the bridge collapse in 2007 in Minnesota, Congress allocated $250 million.

Initial estimates put the cost of rebuilding the bridge at $600 million, according to economic analysis company IMPLAN.

Federal officials have told Maryland lawmakers the final cost of rebuilding the bridge could soar to at least $2 billion, Roll Call reported, citing a source familiar with the discussions.

Insurers could face billions of dollars in claims, analysts said, with one putting the cost at as much as $4 billion, which would make the tragedy a record shipping insurance loss.

HOW LONG WILL IT TAKE TO REBUILD THE BRIDGE?

Rebuilding could be a lengthy process and will depend on whether any of the remaining structure can be salvaged. It took five years to construct the original bridge from 1972-1977.

The closure of the port for just one month would cost Maryland $28 million in lost business, according to IMPLAN.

WHAT SHIP HIT THE BALTIMORE BRIDGE?

The Dali was leaving Baltimore en route to Colombo, Sri Lanka, with 21 crew and two pilots on board.   

The ship measures 948 feet (289 m) - as long as three football fields. It was stacked high with containers but capable of carrying twice as much cargo. Safety investigators recovered the ship's black box, which can give them the vessel's position, speed, heading, radar, bridge audio, and radio communications as well as alarms. 

The same ship was involved in an incident in the port of Antwerp, Belgium, in 2016, when it hit a quay as it tried to exit the North Sea container terminal. 

A later inspection in June 2023 carried out in San Antonio, Chile, found the vessel had "propulsion and auxiliary machinery" deficiencies, according to data on the public Equasis website, which provides information on ships.

The registered owner of the Singapore-flagged ship is Grace Ocean Pte Ltd, LSEG data show. Synergy Marine Group managed the ship, and Maersk chartered the vessel.

WHAT DO WE KNOW ABOUT THE BRIDGE THAT COLLAPSED?

The Francis Scott Key Bridge was one of three ways to cross the Baltimore Harbor and handled 31,000 cars per day or 11.3 million vehicles a year.

The steel structure was four lanes wide and rose 185 feet (56 m) above the river.    

It opened in 1977 and crosses the Patapsco River, where U.S. national anthem author Francis Scott Key wrote the "Star Spangled Banner" in 1814 after witnessing the British defeat at the Battle of Baltimore and the British bombing of Fort McHenry. 

HOW WILL THE BRIDGE COLLAPSE IMPACT THE BALTIMORE PORT?

Traffic was suspended at the port, the 17th largest in the country, and the jobs of 15,000 people are on hold.

The flow of containers to Baltimore can likely be redistributed to bigger ports. However, there could be major disruptions in shipping cars, coal and sugar.

It is the busiest U.S. port for car shipments, handling at least 750,000 vehicles in 2023, according to data from the Maryland Port Administration.

In 2023, the port was the second busiest for coal exports.

It is also the largest U.S. port by volume for handling farm and construction machinery, as well as agricultural products such as sugar and salt.      

(Writing by Lisa Shumaker; Editing by Daniel Wallis and Bill Berkrot)

Copyright 2024 Thomson Reuters .

Join the Conversation

Tags: United States , Sri Lanka , Maryland

America 2024

Health News Bulletin

Stay informed on the latest news on health and COVID-19 from the editors at U.S. News & World Report.

Sign in to manage your newsletters »

Sign up to receive the latest updates from U.S News & World Report and our trusted partners and sponsors. By clicking submit, you are agreeing to our Terms and Conditions & Privacy Policy .

You May Also Like

The 10 worst presidents.

U.S. News Staff Feb. 23, 2024

Cartoons on President Donald Trump

Feb. 1, 2017, at 1:24 p.m.

Photos: Obama Behind the Scenes

April 8, 2022

Photos: Who Supports Joe Biden?

March 11, 2020

Biden Besting Trump in Campaign Cash

Susan Milligan and Lauren Camera March 29, 2024

The Baltimore Bridge Collapse, Explained

Elliott Davis Jr. March 29, 2024

Key Inflation Gauge Ticks Up

Tim Smart March 29, 2024

Who Is ISIS-K?

Trevor Bach March 28, 2024

Consumers Steady as Economy Hums Along

Tim Smart March 28, 2024

'Very Active' 2024 Hurricane Season

Cecelia Smith-Schoenwalder March 27, 2024