a mixed method research study on

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 17 January 2023

A mixed-method study on adolescents’ well-being during the COVID-19 syndemic emergency

• Alessandro Pepe 1 , 2 &
• Eleonora Farina 1 , 2  

Scientific Reports volume  13 , Article number:  871 ( 2023 ) Cite this article

2749 Accesses

5 Citations

6 Altmetric

Metrics details

• Health care

In this study, we set out to investigate adolescents’ levels of perceived well-being and to map how they went about caring for their well-being during the COVID-19 syndemic. Participants were 229 Italian adolescent high school students (48.9% males, mean age = 16.64). The research design was based on an exploratory, parallel, mixed-method approach. A multi-method, student-centered, computer-assisted, semi-structured online interview was used as the data gathering tool, including both a standardized quantitative questionnaire on perceived well-being and an open-ended question about how adolescents were taking charge of their well-being during the COVID-19 health emergency. Main findings reveal general low levels of perceived well-being during the syndemic, especially in girls and in older adolescents. Higher levels of well-being are associated with more affiliative strategies (we-ness/togetherness) whereas low levels of well-being are linked with more individualistic strategies (I-ness/separatedness) in facing the health emergency. These findings identify access to social support as a strategy for coping with situational stress and raise reflection on the importance of balancing the need for physical distancing to protect from infection, and the need for social closeness to maintain good mental health.

Similar content being viewed by others

Adults who microdose psychedelics report health related motivations and lower levels of anxiety and depression compared to non-microdosers

People are surprisingly hesitant to reach out to old friends

Effect of breathwork on stress and mental health: A meta-analysis of randomised-controlled trials

Introduction.

Officially declared a pandemic on 11 March 2020, the COVID-19 outbreak has resulted so far in over 550million cases and 6.3million deaths worldwide 1 . Although it appeared at the start of the pandemic that all populations around the world would be affected to the same extent, we now know that this is not the case. Indeed, while multiple sources initially claimed that "we were all in the same boat," two years after the onset of the pandemic, we are now in a position to state that "We are all on the same sea, but the boats from which we are dealing with the effects of COVID-19 are very different." Not only have the more strictly medical aspects differentially affected different populations (with the outcome of exacerbating inequalities), but the measures implemented by various governments for reducing the spread of the disease (e.g., lockdowns, restrictions on movement, school closures, and the adoption of distance education) have differentially affected different segments of the population in each country 2 .

Covid-19 from a syndemic perspective

Variability in the evidence reported in the literature regarding the effects of the COVID-19 pandemic on different populations of interest and in different contexts may be explained by drawing on the concept of syndemic 3 . A syndemic is a situation in which two or more health conditions co-occur in environments of aggravated adversity and interact synergistically to yield worse health outcomes than each affliction would likely generate on its own 4 . Limiting the harms caused by COVID-19 will require paying far greater attention to so-called noncommunicable diseases (NCDs) and socioeconomic inequality than has been done up to now. Although NCDs have conventionally been analyzed in relation to the risk factors for cardiovascular diseases, cancers, chronic pulmonary diseases, and diabetes, scholars have recently emphasized 5 that cardiovascular diseases, diabetes, cancer, and respiratory diseases frequently co-occur with both common mental disorders (such as depression and anxiety) and severe mental illnesses (such as schizophrenia and bipolar disorder). A syndemic is more than the outcome of a pandemic in terms of comorbidities; rather, it is an intertwining of biological and social conditions that increases an individual's susceptibility to harm or worsens their health outcomes. The most important implication of viewing the COVID-19 outbreak as a syndemic is that this helps to focus on its social origins. The vulnerability of younger and older citizens, ethnic minority communities, and key workers, who are frequently underpaid and enjoy less social welfare protection, points to an unacknowledged truth: no matter how effective a treatment or protective a vaccine, any exclusively biomedical solution for COVID-19 will fail.

Impact of syndemic on adolescents’ well-being and mental health

The international scientific literature presents extensive research on the effects of the syndemic on individual well-being in different age groups and based on different methods of inquiry 6 . Adolescents, although at lower risk of death or severe illness due to COVID-19 than the adult population, are still having to cope at different levels with the negative impact of the public health emergency on their mental health 7 . Among other manifestations, the literature highlights anxiety-related, depressive, psychosomatic symptoms, as well as high levels of post-traumatic stress; these symptoms are more marked in girls, older adolescents, and adolescents with pre-existing vulnerabilities 8 . A recent review of 156 studies on changes in adolescents’ mental health during the COVID-19 emergency showed that outcomes had significantly worsened in several areas 9 . Among studies of depression, some 79% of studies found that participants’ symptoms had worsened, while 76% of studies of anxiety identified a worsening of symptoms, especially in girls and young women. Similarly, 70% of the research on stress and distress observed a clear increase in these phenomena with respect to the pre-pandemic period. Also considered in the review were studies—both longitudinal and cross-sectional—that found changes in subjective well-being, quality of life, and life satisfaction: the vast majority of authors identified a worsening of these dimensions. On the other hand, contrary to fears that adolescents would engage in greater substance abuse during the COVID-19 health emergency, findings regarding the use of substances have been mixed. A recent report by the U.S. National Institutes of Health identified a sharp decline in adolescent substance use in 2021 10 . The syndemic’s impact on substance use has likely been moderated by a number of factors, including changes in the social settings in which young people normally have the opportunity to use substances. In-person social interaction was drastically reduced in many contexts, thus reducing opportunities to drink alcohol 11 . In contrast, other studies showed that young people who remained more isolated during stay-at-home regimes used more cannabis than those who continued to socialize in person 12 .

However, in relation to mental health problems, contrasting results have been found both within and between categories. The overall decline in mental health is likely related to multiple factors implicated in the COVID-19 emergency internationally, including the specifics of different socioeconomic backgrounds 13 as the concept of syndemic also reflects. Although most longitudinal studies on depression, anxiety and stress have documented an increase in symptoms over the period of the COVID-19 emergency, others have found no change or even a decrease in the incidence of these symptoms. Increases in suicide ideation and suicide attempts have been reported in several countries, but in one of the few studies conducted with a subgroup of marginalized youth, a significant reduction in episodes of self-harm was reported during the pandemic, potentially attributable to good service response 14 . Furthermore, different aspects of the syndemic setting likely generated different mental health problems. For example, a survey of U.S. students found that school-related concerns (e.g., lower quality online courses) were associated with increased depressive symptoms, while concerns related to home confinement per se (e.g., “cabin fever”) were associated with increased generalized anxiety symptoms 15 . It is also worth reporting the various studies during these syndemic years that also noted people's ability to detect positive aspects related to the emergency situation. For example, one line of research focused on the development of a form of wisdom derived from the ability to detect positive opportunities (such as spending more time with family members, developing forms of solidarity with other people) to develop new attitudes, behaviors, and values 16 . Other studies have pointed out that the emotional impact of syndemia, although predominantly characterized by emotions such as anxiety, sadness, and fear, over time also brings out positive emotions such as hope, trust, and tranquility 17 .

Maintaining personal well-being during the syndemic

Studies that did not find significant changes in levels of well-being among youth have identified an association with the use of positive coping strategies in many cases 18 , 19 . Young people report that they used different ways to sustain resilience: in most cases, these strategies involved trying to maintain relational connections with significant others who were physically out of reach (friends and relatives), combined with more individual approaches to maintaining physical and mental well-being (exercising, spending time outdoors, meditating…).

Among the different factors that can reduce the risk of non-communicable disease during a syndemic, the literature recognizes coping strategies, along with perceived social support, as protective against the development of acute symptoms following exposure to particularly stressful events. Coping is the deployment of behavioral and cognitive strategies to modify negative aspects of one’s environment, and to minimize or escape internal threats induced by stress or trauma. Such strategies are diverse and can be more or less adaptive 20 , 21 . A more active coping style includes problem-oriented strategies, for example target the context as a means of solving difficulties, create an action plan, referring to someone, and be free to express and share feelings. Avoidance strategies include denial, substance use, and behavioral and mental detachment: trying to suppress emotions, withdrawing from people and enacting risky behaviors can be examples of avoidant coping style. Social support can be sought with a view to acquiring understanding or information or as an emotional outlet, which is a crucial resource to cope with stressful events and develop a positive attitude of acceptance, containment, and positive reinterpretation of events. In literature emerged that the use of avoidant coping strategies among adolescents was associated with overall higher levels of anxiety and depression and with other factors related to living conditions, such as having three or more siblings, having separated parents with low educational level 22 .

The restrictions imposed in the context of the COVID-19 health emergency have drastically reduced and disrupted access to many forms of social support, meaning that one coping strategy is less available or completely unavailable. However, studies show that family life during the initial severe lockdown of 2020, although it severely constrained adolescents’ drive for autonomy—hindering the fulfillment of a fundamental developmental task—acted as a key protective factor in their mental well-being 23 , 24 .

In light of the strong association between adolescents’ interactions with peers, friends, and family and their psychological well-being, it is of crucial importance to examine the factors that could further hinder or damage interpersonal interactions during this vulnerable stage of life.

The present study

In this study, we set out to investigate adolescents’ levels of perceived well-being and to map how they went about caring for their well-being during the COVID-19 syndemic. In keeping with the literature reviewed above, we hypothesized that they would have drawn on both individual and social resources to feel good and safe, as well as making novel use—and possibilities to use—of indoor and outdoor spaces. We expected that different strategies would be associated with differential levels of well-being. More specifically, we hypothesized that a strategy of seeking to maintain satisfying and supportive relationships with family and peers would foster the deployment of more proactive coping attitudes and consequently higher levels of perceived well-being. In contrast, we predicted that more individualistic and inward-looking coping strategies would be associated with a tendency toward passivity, a diminished perception of being in control of the situation, and consequent lower levels of well-being.

Participants were 229 Italian adolescent high school students. The sample was balanced in terms of gender, comprising 48.9% males (n = 112) and 48% females (n = 110); seven participants (3.1%) did not specify their gender. Participants’ ages ranged from 14 to 19 years ( M  = 16.64, SD  = 1.46). The inclusion criteria were: (1) attending high school, (2) being aged between 14 and 19 years, (3) accepting the terms of participation in the research. We did not apply any exclusion criteria. We recruited a convenience sample via a non-probability sampling technique whereby participants are selected from the population only because they agree to participate 25 . We collected the data during the period from April 2021 to June 2021.

Procedure and materials

This exploratory study was underpinned by a parallel mixed-method research design 26 and its primary source of data was a multi-method, student-centered, computer-assisted web interview (CAWI) 27 , 28 . The research protocol comprised three main sections: (1) demographic background, (2) closed items about well-being, (3) two open-ended question about being an adolescent during the COVID-19 public health emergency (“At this time, what are the times, situations, or events that help you feel good?” and “If you were to describe, using a phrase, image, or metaphor, what it is like to be a girl/boy of your age these days, what would you write?”). With regards to demographic data, the research plan included age and gender as variable of interest since the study was exploratory and used a convenience sample. Data were collected anonymously, and all participants were briefed about the research aims and procedure. Participation in the study was on a voluntary basis, meaning that participants received no monetary or financial rewards. The study was approved by the Ethics Board at Milano-Bicocca University (prot. N. 0059806/21) and was conducted in keeping with the ethical principles laid down in the Declaration of Helsinki 29 and the American Psychological Association code of conduct 30 . Informed consent was obtained from all participants and from parents for underage participants. During data collection (i.e., April to June 2021, a zoning policy was still in effect, based on the rate of contagion, while secondary school students were attending in-person classes 50% to 100% of the time, depending on the zone and the internal organization of schools.

World Health Organization Well-Being Index (WHO-5) The five-item World Health Organization Well-Being Index (WHO-5) is a short rating scale measuring global subjective well-being 31 . The instrument has been used in many different settings to assess positive well-being and as a proxy for mental health 32 . The questionnaire items are: (1) ‘I have felt cheerful and in good spirits’, (2) ‘I have felt calm and relaxed’, (3) ‘I have felt active and vigorous’, (4) ‘I woke up feeling fresh and rested’ and (5) ‘My daily life has been filled with things that interest me’. Respondents rate each item on a Likert scale ranging from 5 (all of the time) to 0 (none of the time). The raw WHO-5 scores are computed by summing the scores for the individual items, yielding global scores ranging from 0 (no well-being) to 25 (maximal well-being) which are then conventionally converted to a scale of 0–100. A generally accepted threshold for poor well-being and the risk of developing depressive symptoms is less than 50 33 . In this study, Cronbach’s alpha reliability coefficient (α) 34 was 0.817.

Qualitative material In line with our research aims, we analyzed the open-ended question “At this time, what are the times, situations, or events that help you feel good?” with a view to gathering direct information about how adolescents tried to taking care of their well-being during the COVID-19 health emergency. A total of 223 responses were collected, totaling 1915 words, with an average response length of 8.6 words. In terms of missing values, 6 participants did not respond or responded "I don't know," resulting in a missing value rate of around 2.5%.

Data analysis strategy

We analyzed the data from our mixed-method questionnaire using quantitative textual analysis (QTA) 35 . QTA is a form of qualitative content analysis and assumes that (1) words that tend to appear together (i.e. close proximity) in a given context may be interpreted as related to a common lexical theme or concept within the discourse under study 36 and (2) traditional statistical techniques may be used to analyze narrative data 37 . Hence, we analyzed the adolescents’ replies to the question “If you were to describe, using a phrase, image, or metaphor, what it is like to be a girl/boy of your age these days, what would you write?” via co-word analysis of correspondence based on our research interests (CA) 38 . The advantage of using CA to analyze this kind of material is that this method allows the researcher to examine the structure of a dataset by rescaling a set of proximity measures into visual distances representing specific locations in a spatial (Cartesian coordinate system) configuration 39 . The analysis yields word-maps which allow the researcher to identify recurring themes, their degree of salience, and how they relate to one another. We assessed similarities via the chi-square and Salton’s cosine indexes 40 along with their statistical significance (set at p < 0.05). Salton’s cosine allows us to organize the relations geometrically so that they can be visualized as structural patterns of relations 41 .

To make the results of the co-word analysis more understandable, word-based concept mapping tools based on multivariate QTA methodologies may be used to identify dominant themes, their relative weight, and how they relate to one another within a given set of textual data 42 . Many studies in the field of health psychology and health promotion 43 , 44 have suggested that common cluster analysis of textual data may be an interesting solution when researchers wish to gain meaningful insight into participants' words by bringing a positivist approach to bear on qualitative data 45 . In the present study, we used the k-means cluster analysis algorithm 46 . The k-means algorithm first groups objects into an arbitrary number of clusters, then computes cluster centroids and assigns each object in such a way that the squared error between it and the empirical mean of a cluster is minimized (i.e., Euclidean distance is used). K-means, like other techniques, seeks to minimize variability within clusters while maximizing variability between clusters 47 . A second critical issue in performing k-means cluster analysis in exploratory QTA is determining the optimal cluster configuration, where optimal refers to the outcome among all possible grouping combinations that presents the full set of the most meaningful associations 48 . Determining what distribution of clusters provides a better understanding of data requires the selection of an objective 'measure of optimal partitioning' (or clustering validity criteria). We chose Calinski- Harabasz index, which is also known as the Variance Ratio Criterion (VRC) 49 from among the available measures because it evaluates the quality of data partitions according to a standard formula. Specifically, the greater the value of the between variance-within variance ratio normalized with respect to the number of clusters, the superior the data partition. To find the best configuration, we ran cluster analysis on the word co-occurrence matrix with varying numbers of clusters (from three to nine), choosing the solution with the best local VRC peak. For all analyses, the alpha level was set at 0.05. In the context of the present study, we expected that the output of the CU would allow us to analyze adolescents’ chosen metaphors by grouping “naturally” occurring emerging themes as a function of lexical similarity and in relation to well-being scores. All analyses were conducted using TLAB 5.0 and SPSS 21.0.

Data cleaning and general descriptive statistics for the lexical corpus

As with other data exploration techniques, QTA required a pre-processing stage to prepare data for analysis. As recommended in the literature, we conducted normalization (removing all general function words such as articles, connection forms, and prepositions); lemmatization (reducing all inflected words to their root form as found in the dictionary) and synonimization (reducing words that may be considered equivalent from the semantic point of view—e.g., illness and sickness—ì to the same root form) with a view to preserving the accuracy of the textual data and preparing the database for running algorithms designed to generate both occurrence and co-occurrence matrices (for details about the process, see 50 , 51 , 52 ).

The resulting qualitative database comprised 1616 occurrences, 652 raw forms, and 439 hapaxes (i.e., words that occurred only once in the text). By adopting a threshold of at least four occurrences (text coverage 83%), root type/token ratio (an index of text richness 53 ) was 16.21, suggesting that the data were suitable for multiple correspondence analysis.

The results are divided into two sections. In the first section, we present quantitative data (i.e., descriptive statistics and zero-order correlations) concerning the levels of general well-being recorded in the adolescent sample using the World Health Organization threshold. In the second section, we summarize the results of the co-word correspondence analysis and subsequent clustering procedure.

The quantitative data outcomes are reported in Table 1 .

The zero-order correlations suggest that the adolescents’ levels of well-being were negatively associated with age and gender, with younger participants reporting greater well-being and girls (WHO-5 mean score = 47.6) reporting less happiness than boys (WHO-5 mean score = 57.9). In this regard, analysis of variance revealed that the difference in levels of well-being between gender-based groups was statistically significant [t(1,220) = 3.54, p 0.001]. In addition, 36.6% of boys and 54.5% of girls obtained scores of less than 50, indicating that they were at risk of developing depressive symptoms. Furthermore, 12.5% of boys and 20.9% of girls scored less than 28, suggesting that they were at risk of clinical depression. Before moving on to the qualitative analyses, in Fig.  1 we show the percentages of boys and girls classified according to the World Health Organization’s well-being spectra. This figure illustrates the differences in well-being scores between boys and girls, particularly in the group at risk of developing clinical symptoms of depression and the group reporting high well-being.

Adolescents grouped according WHO5 scores. Participants with scores of under 28 were at risk of clinical depression, while those with scores of over 75 displayed high levels of well-being.

The first result of the QTA concerned the words most frequently used by the cohort of adolescents to describe the moments, situations, or events that help them feel good during the COVID-19 syndemic. Given that frequently occurring words reflect recurring themes in a textual corpus and serve as the foundation for more complex coding categories, this is a preliminary form of analysis. The most frequently occurring words in the data set (with the number of occurrences reported in brackets) were: friends (137), family (51), to hang out (38), sport (18), music (18), boyfriend (16), to play (15), time (15), to meet (10), to chat (9), to watch (8), to listen (8), home (7), to help (7), people (6), on-line (5), to sleep (5) and alcohol (4). Even this initial look at the data provides some insight into the contents of adolescents’ strategies for coping with difficulties related to the syndemic; however, this level of interpretation is still quite biased (e.g., word frequency count is not weighted in relation to the length of responses), and it does not reveal the underlying structures in the data or the associations between words. When cluster analysis is used, it provides a more detailed picture. Because the evidence reviewed in the literature does not provide a theoretical framework for the structure of coping strategies, we began our exploratory analysis by determining the most appropriate cluster configuration for our qualitative data. Table 2 displays the values obtained for this purpose via the Calinski-Harabasz index.

The VRC values revealed that, based on the defined word co-occurrence matrix, the optimal configuration was a solution with four distinct clusters. Peak VRC was found to explain 66.2% of total variance, with low within-variance values: cluster_1 (CL1, ssw = 0.162), cluster_2 (CL2, ssw = 0.131), cluster_3 (CL3, ssw = 0.081) and cluster_4 (CL4, ssw = 0.073). In terms of cluster density, the partition of words across the clusters was relatively even and satisfactory, with CL1 including 35.1% of replies and CL2, CL3 and CL4 including 22.8%, 31.9% and 10.1%, respectively. We then investigated the main contents of the coping strategies adopted by adolescents by calculating the association between the replies grouped in each cluster and the cluster itself (in terms of distance from the centroids). In addition, we evaluated the associations between variables (e.g. age, gender and levels of well-being) and clusters by calculating χ 2 and its statistical significance. Finally, we plotted the cluster coordinates in two-dimensional factorial space to bring to light the meaning of the individual factors. Figure  2 offers a graphical representation of the four-cluster solution.

Graphical representation of clusters and occurrences in a Cartesian Space.

Cluster 1: In general, this cluster was associated with boys (χ 2  = 6.35, p = 0.012) aged 16 years (χ 2  = 5.85, p = 0.016) who reported a high level of well-being (χ 2  = 6.31, p = 0.012). Quotes from this first cluster include: “ hanging out with friends, family, music ” (Boy, 16 y.o, WHO5 = 64), “ the rare evenings when I get together with my group of friends to quietly play some board games ” (Boy, 17 y.o., WHO5 = 72), “ being with my friends, going out, and talking to people close to me .” (Girl, 16 y.o., WHO5 = 64) and “ being with my friends ” (Girl, 16 y.o., WHO5 = 72).

Cluster 2: This cluster grouped 18-year-old adolescents of (χ 2  = 11.85, p = 0.001) with low levels of well-being (χ 2  = 9.42, p = 0.002). Representative quotes included in the cluster were: “ Being with my family or boyfriend, or practicing sports ” (Girl, 18 y.o., WHO5 = 44), “ being with myself ” (Boy, 18 y.o., WHO5 = 38), “ Resting, going out, shopping ” (Male, 16 y.o., WHO5 = 24), “ I listen to music very often, in the evening I sometimes spend time on video calls with "friends" I met online who are from different countries ” (Male, 16.y.o, WHO5 = 36).

Cluster 3: this cluster was associated with 17-year-old adolescents (χ 2  = 4.60, p = 0.032) with a medium–low level of well-being (χ 2  = 4.11, p = 0.042). Quotes include: “ Nighttime, when all is silent and the thoughts screaming in the head fly away ” (Male, 19 y.o., WHO5 = 44), “ sport, alcohol, family ” (Male, 18 y.o., WHO5 = 40), “ Making music, being with my family, and going out for leisurely walks ” (Girl, 17 y.o., WHO5 = 54) and “ I do well in class, in the afternoon I never go out except sometimes with only one friend, so being in class with my classmates makes me feel good because there is no need for me to arrange to be with them ” (Male, 15 y.o., WHO5 = 48).

Cluster 4: The final cluster, which was the least dense accounting for approximately 10% of responses, was exclusively associated with “older” participants aged 19 years (χ 2  = 16.18, p = 0.032). In this case, representative quotes included: “ Seeing my mother happy, feeling right with myself ” (Male, 19 y.o., WHO5 = 44), “ Dancing ” (Girl, 16 y.o., WHO5 = 60), “ Playing online games, watching anime, and talking with friends ” (Male, 17 y.o., WHO5 = 76) and “ taking advantage of my free time ” (Male, 18 y.o., WHO5 = 56).

Before proceeding with the last step in the data analysis, we deemed it of interest to list all the coping strategies deployed by the adolescents with a WHO-5 well-being score of under 28: “ leisure time”, “my family”, “no one”, “Being with friends”, “listening to music and playing with the Xbox”, “seeing my friends outside of school”, “seeing friends and sleeping”, “Seeing my friends and playing football with my team”, “being with friends”, “spending time with my friends”, “there is no time”, “going to school”, “I have no idea”, “sleeping and eating” and “resting, going out, shopping ”. With a view to comparison, we similarly listed all the coping strategies drawn on by the adolescents with a WHO-5 well-being score of over 75: “ My friends, soccer, and my girlfriend's love”, “My family and my girlfriend”, “Family and friends”, “Being with friends”, “Being with family during the holidays”, “My parents' affection and my friends' trust”, “When I am with friends, when I am with my family at home or outside”, ”hanging out with friends”, “being with the people who make me happy, friends and family”, “hanging out with friends and family”, “family, hanging out with friends and playing soccer”, “soccer”, “sports, hanging out with friends and playing” and “Being with people who love me”. It should be noted here that there are substantial differences in the well-being strategies described by these two groups of adolescents (high vs. low well-being). In the descriptions of the high well-being group, for example, both the actions taken to feel better and the different social actors (friends, family members, boyfriends) involved, as well as the positive emotions and feelings felt (love, affection, happiness), were also present. This component (positive emotion and feelings) was lacking in the descriptions of the group with low well-being.

The final step in our data analysis was to label the two axes of Cartesian space with a view to defining a framework of meaning within which to organize the clusters. The principal axis is the straight line that runs closest to the profile point and passes through the zero point, hence meaning is identified first along the y -axis and then along the x -axis (Fig.  2 ). Conventionally, this type of graph in QTA cluster analysis is interpreted in terms of the geometric figures that can be drawn between the representation’s outermost points (see 35 ): in this case, the “triangle” drawn between CL1 (center-right), CL4 (top-left and bottom-center), CL3 (bottom-left). Looking at the first axis (X), we can see that this dimension has two poles: the negative extreme to the left is constituted by CL4 and CL3, whereas the positive extreme to the right is CL1 (the terms positive and negative are only artifacts of the calculus process, and they could easily be inverted within this framework). CL1 was associated with high levels of well-being, while CL4 and CL3, at the opposite pole, grouped adolescents reporting low or medium–low levels of well-being. Consequently, the x dimension may be labeled level of well-being. Similarly, the second axis ( y ) divides CL4 at the positive extreme from CL3 at the negative extreme, with CL2 and CL1 remaining in the middle. CL4 (and CL2, whose projection on the Cartesian axis is very close to that of CL4) included coping strategies that may be conducted alone or with a limited number of people, such as: dancing, playing online games, shopping, and making music. On the other side, CL3 (and CL1, whose projection on the Cartesian axis resembles that of CL2) seemed to group coping strategies that were more social and collective in nature, such as: spending time with family and friends. Hence, the y -axis may be labeled as a second dimension of coping strategies that reflects a notion of I-ness as opposed to a sense of We-ness 54 .

To summarize our findings from this QTA of qualitative data collected from adolescents during the COVID-19 syndemic and integrate them into the existing framework of coping strategies for well-being, the cluster analysis results implies the existence of two 'macro-dimensions' that allow us to organize otherwise apparently “atomized” elements of subjective experience during a time of health emergencies and existential uncertainty. A first factor termed level of well-being and a second termed i-ness/we-ness. In this sense, coping strategies of adolescent during COVID-19 syndemic seem not only to range from individuation (I-ness) to affiliation (we-ness)—or, to draw on the words of Wiekens and Stapel, from a sense of togetherness (We-ness) to a sense of separateness (I-ness); rather, they also seem to be strongly associated with different levels of well-being.

The aim of the present study was to advance our understanding of adolescents’ perceived personal well-being during the COVID-19 public health emergency. Besides assessing participants’ levels of well-being by means of a validated and standardized instrument such as the WHO-5, we were interested in exploring the way adolescents were taking care of their mental health during the syndemic period. Our results were generally in line with the hypotheses that we had formulated, offering insights into how best to support adolescents’ mental health trajectories, informing a complex interpretation of the concept of wellbeing, and calling for further, more in-depth investigation.

The role of age and gender in adolescents’ wellbeing

Looking at the results for gender and age differences in relation to perceived well-being, our data support the findings already reported in the literature: girls perceive significantly lower levels of well-being than their male peers 55 , 56 . In addition, older ages are correlated with lower levels of well-being with a general decline in mental well-being with increasing age, whereby older adolescents experience lower levels of life satisfaction, are less likely to report excellent health, and suffer more frequent mental health problems 57 . Furthermore, the same study showed that, by age 15, girls report poorer mental well-being than boys. The COVID-19 syndemic has confirmed and in some cases accentuated these differences, with older female adolescents suffering more from anxiety and depressive symptoms 58 . This situation is part of a broader picture whereby adolescent mental health has been undergoing a general decline in recent years 59 : for example, a study 60 identified, from 2018 to 2020, a decrease in mean perceived well-being, as measured by the WHO-5, from 43.7 to 35.8 (albeit that both of these scores invite reflection on the state of mental health in adolescence more generally). Thus, it seems that the COVID-19 emergency has accelerated a process that had already been underway for some years, and which requires policy makers to urgently examine the adequacy of current mental health promotion services and practices.

Self-care practices

The results of the QTA offer us a more in-depth and nuanced understanding of the conditions that influence adolescents’ well-being, including the role of gender and age. If we examine cluster 1, we find mainly younger male adolescents, who, when asked the open-ended question about how they take care of their personal well-being, answer by naming strategies chiefly aimed at maintaining peer relationships and teams sports-playing. This cluster is associated with high levels of well-being. On the other hand, in clusters where levels of well-being are lower, adolescents refer to the use of more individualistic and intimate, but also more "passive" strategies (music, shopping…). When looking at the coping strategies reported by those with "extreme" scores on the well-being curve (under 28 and over 75), key differences emerge. Adolescents with scores of < 28 (who are thus potentially at risk of depression) report seeking support from relationships, yet positive affective states rarely feature in their responses, and they make greater use of "static" verbs ("I'm with," "I see…"). Furthermore, in the responses of the group of adolescents with very low levels of well-being, food or alcohol intake (about 5%) also appeared as—dysfunctional—coping strategies, along with higher levels of apathy ("sleep longer"…). In contrast, adolescents with high levels of perceived well-being reported actively seeking out social support and positive relationships, within their families and among their friends, and these efforts were more frequently and explicitly associated with affective and emotional states. This finding corroborates studies in the literature which suggest that adolescents’ growing desire for autonomy and independence from parents and to belonging to a peer group 61 , 62 takes shape in parallel with the maintenance of close and positive relationships with family as a key requirement for psychological well-being and adjustment 63 . In particular, adolescents who have poorer and dysfunctional family interactions and relationships experience greater psychological maladjustment. In the context of the public health emergency, everyday living conditions, especially the fact of sharing the same spaces with family for a prolonged length of time, likely amplified the impact of positive vs dysfunctional family relationships on levels of well-being. We might speculate that the participants in the present study who most frequently mentioned their family as a positive resource are those who were already embedded in more protective and functional systems. Nevertheless, examining the levels of well-being of clusters 1 and 2 (medium–high) versus cluster 3 (medium–low), it seems that it is the combination of family-friends (as in cluster 1 and 2) as sources of support, as opposed to "just" family (as in cluster 3), that makes the difference with respect to levels of perceived well-being.

When the dimensions of well-being and coping strategies are jointly represented along Cartesian axes, a continuum emerges from high levels of well-being associated with more affiliative strategies (we-ness/togetherness) to low levels of well-being associated with more individualistic strategies (I-ness/separatedness). Again, the collective dimension emerges as a resource. However, this result prompts reflection about access to social support as a strategy for coping with situational stress. Thinking "we are all on the same sea" is a view that may help, but it is also true that "everyone has a different boat": those who experience positive and satisfying family and extra-familial relationships may be more likely to identify and seek out the collective dimension as a potential source of protection against stress, while those who had already been experiencing conditions of marginality or dysfunctional family relationships or vulnerability prior to the advent of COVID-19 may find it more difficult and/or unhelpful to turn to more "social" coping strategies. We might reflect on how much this syndemic has widened such gaps, which are not only economic but also social and political, among people in general and among adolescents in particular. Studies have proven that in fragile adolescents (suffering from anxiety and depression), the impact of the public health emergency has further exacerbated their situation and increased the distance between these youths and peers with good levels of mental health who are well integrated at the socio-relational level: indeed, the literature shows that adolescents who experience greater symptoms of anxiety and depression experience a deterioration in social well-being over time, and receive less social support and greater victimization from peers 64 . Other similar studies have shown how the COVID-19 outbreak and the related risk-reduction strategies have changed the social contexts of adolescents in low- and middle-income countries, with profound implications for their well-being, especially in the case of vulnerable adolescents, including those affected by poverty and armed conflict. In such contexts, pre-existing conditions of disadvantage have had a negative impact on the mental health of adolescents, especially that of girls 65 .

Thus, in the current syndemic setting, the issue of "non-communicable disease" emerges strongly, coupled with, and exacerbating the impact of the virus on the physical health of self and significant others. In this scenario, it is not difficult to discern whether the increase in mental health symptoms is the result of the disease itself (directly or indirectly experienced, or as a source of concern for one’s own safety) or the related restrictive measures (e.g., separation from friends, disruption of school, etc.). In any case, it is crucial to carefully weigh the potential benefits of reduced COVID-19 transmission against the detrimental effects on mental health of social isolation, especially in adolescents. From this perspective, insistent calls for social distancing from the authorities seem unfair and counterproductive: while physical distancing may offer protection from a physical health perspective, we need social closeness to maintain good mental health.

Limitations

This study, like others of its kind, features several limitations that should be noted. First, the study is cross-sectional, which means that the teenagers were questioned at a point in time when the COVID-19 epidemic was still ongoing. While this was congruent with the research aims, the research design offers no information about the dynamic evolution of the phenomena under observation, either in terms of well-being or in terms of self-care strategies. A second limitation concerns the fact that the interviews were administered online. Although this approach facilitated data collection at a time when mobility constraints and public health measures made it difficult to gather data directly in the field, it raises concerns regarding the sample's effective representativeness. The most susceptible groups of teenagers or those with educational and economic difficulties may have had restricted access to the Internet and computer technologies, affecting their ability to respond the survey. This means that caution is required in generalizing our findings to all Italian teenagers. Another limitation is that the research plan only considered demographic variables such as age and gender. Instead, studies are needed to detect the effects of other contextual variables that may be associated with adolescent well-being in order to better assess the dynamics of syndemics. In the future, follow-up studies within this line of inquiry should be conducted with larger samples and longitudinal designs, in order to gain a clearer picture of the variables studied, in terms of both the stability of the identified associations and the scope for change, especially given the fact that adolescence in general is a period of transition and rapid transformation.

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Health Organization, W. Infection Prevention and Control in the Context of Coronavirus Disease (COVID-19): A Living Guideline . https://apps.who.int/iris/bitstream/handle/10665/353565/WHO-2019-nCoV-ipc-guideline-2022.2-eng.pdf?sequence=1 . (2022).

Blundell, R. et al. Inequality and the COVID-19 Crisis in the United Kingdom. Annu. Rev. Econ. 14 , 607–636 (2022).

Article   Google Scholar  

Horton, R. Offline: COVID-19 is not a pandemic. Lancet 396 , 874 (2020).

Article   CAS   Google Scholar  

Willen, S. S., Knipper, M., Abadía-Barrero, C. E. & Davidovitch, N. Syndemic vulnerability and the right to health. Lancet 389 , 964–977 (2017).

Stein, D. J. et al. Integrating mental health with other non-communicable diseases. BMJ 364 , l295 (2019).

de Figueiredo, C. S. et al. COVID-19 pandemic impact on children and adolescents’ mental health: Biological, environmental, and social factors. Prog. Neuropsychopharmacol. Biol. Psychiatry 106 , 110171 (2021).

Bhatia, R. Editorial: Effects of the COVID-19 pandemic on child and adolescent mental health. Curr. Opin. Psychiatry 33 , 568–570 (2020).

Magson, N. R. et al. Risk and protective factors for prospective changes in adolescent mental health during the COVID-19 pandemic. J. Youth Adolesc. 50 , 44–57 (2021).

Zolopa, C. et al. Changes in youth mental health, psychological wellbeing, and substance use during the COVID-19 pandemic: A rapid review. Adolesc. Res. Rev. 7 , 161–177 (2022).

Capasso, A. et al. Increased alcohol use during the COVID-19 pandemic: The effect of mental health and age in a cross-sectional sample of social media users in the U.S.. Prev. Med. 145 , 106422 (2021).

White, H. R., Stevens, A. K., Hayes, K. & Jackson, K. M. Changes in alcohol consumption among college students due to COVID-19: Effects of campus closure and residential change. J. Stud. Alcohol Drugs 81 , 725–730 (2020).

Bartel, S. J., Sherry, S. B. & Stewart, S. H. Self-isolation: A significant contributor to cannabis use during the COVID-19 pandemic. Subst. Abus. 41 , 409–412 (2020).

Adegboye, D. et al. Understanding why the COVID-19 pandemic-related lockdown increases mental health difficulties in vulnerable young children. JCPP Adv. 1 , e12005 (2021).

Kasinathan, J. et al. Keeping COVID out: A collaborative approach to COVID-19 is associated with a significant reduction in self-harm in young people in custody. Austr. Psychiatry 29 , 412–416 (2021).

Hawes, M. T., Szenczy, A. K., Klein, D. N., Hajcak, G. & Nelson, B. D. Increases in depression and anxiety symptoms in adolescents and young adults during the COVID-19 pandemic. Psychol. Med. 2021 , 1–9 (2021).

CAS   Google Scholar  

Flebus, G. B., Tagini, A., Minonzio, M., Dushku, E. & Crippa, F. The Wisdom acquired during emergencies scale-development and validity. Front. Psychol. 12 , 713404 (2021).

Pepe, A., Biffi, E. & Farina, E. Feeling the emotions, finding the resources: A pathway toward balanced parenting?. Aula Abierta 50 , 807–814 (2021).

Hussong, A. M., Midgette, A. J., Thomas, T. E., Coffman, J. L. & Cho, S. Coping and mental health in early adolescence during COVID-19. Res. Child. Adolesc. Psychopathol. 49 , 1113–1123 (2021).

van de Groep, S., Zanolie, K., Green, K. H., Sweijen, S. W. & Crone, E. A. A daily diary study on adolescents’ mood, empathy, and prosocial behavior during the COVID-19 pandemic. PLoS ONE 15 , e0240349 (2020).

Zeidner, M. & Endler, N. S. Handbook of Coping: Theory, Research, Applications (Wiley, 1995).

Google Scholar  

Sica, M. G. & Altoè, M. Coping orientation to problems experienced-nuova Versione Italiana (COPE-NVI): uno strumento per la misura degli stili di coping. Psicoter. Cogn. Comport. 14 , 27 (2008).

Turk, F., Aykut, K. & Kilinç, E. Depression-anxiety and coping strategies of adolescents during the Covid-19 pandemic. Turk. J. Educ. 10 , 58–75 (2021).

Fioretti, C., Palladino, B. E., Nocentini, A. & Menesini, E. Positive and negative experiences of living in COVID-19 pandemic: Analysis of Italian adolescents’ narratives. Front. Psychol. 11 , 599531 (2020).

Pigaiani, Y. et al. Adolescent lifestyle behaviors, coping strategies and subjective wellbeing during the COVID-19 pandemic: An online student survey. Healthcare (Basel) 8 , 472 (2020).

Emerson, R. W. Convenience sampling, random sampling, and snowball sampling: How does sampling affect the validity of research?. J. Vis. Impair. Blind. 109 , 164–168 (2015).

Östlund, U., Kidd, L., Wengström, Y. & Rowa-Dewar, N. Combining qualitative and quantitative research within mixed method research designs: A methodological review. Int. J. Nurs. Stud. 48 , 369–383 (2011).

Creswell, J. W. & Plano Clark, V. L. Designing and Conducting Mixed Methods Research (SAGE Publications, 2017).

Couper, M. P. & Hansen, S. E. Computer-Assisted Interviewing. Handbook of Interview Research (Sage, 2002).

World Medical Association. Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA 284 , 3043–3045 (2000).

Knapp, S. & Vande-Creek, L. A principle-based analysis of the 2002 American Psychological association ethics code. Psychother. Theory Res. Pract. Train. 41 , 247–254 (2004).

Topp, C. W., Østergaard, S. D., Søndergaard, S. & Bech, P. The WHO-5 well-being index: A sistematic review of the literature. Psychother. Psychosom. 84 , 167–176 (2015).

Hoffman, C. J., Ersser, S. J. & Hopkinson, J. B. Effectiveness of mindfulness-based stress reduction in mood, breast-and endocrine-related quality of life, and well-being in stage 0 to III breast cancer: Randomized, controlled trial. J. Clin. Oncol. 30 , 1335–1342 (2012).

Jahoda, M. The psychological meaning of various criteria for positive mental health. In Current Concepts of Positive Mental Health vol. 136 22–64 (Basic Books, 2006).

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika 16 , 297–334 (1951).

Article   MATH   Google Scholar  

Jason, L. & Glenwick, D. Handbook of Methodological Approaches to Community-based Research: Qualitative, Quantitative, and Mixed Methods (Oxford University Press, 2016).

Brier, A. & Hopp, B. Computer assisted text analysis in the social sciences. Qual. Quant. 45 , 103–112 (2011).

Miller, M. & Riechert, B. P. Frame mapping: A quantitative method for investigating issues in the public sphere. Progress Commun. Sci. 2001 , 61–76 (2001).

Greenacre, M. & Blasius, J. Multiple Correspondence Analysis and Related Methods (Chapman and Hall/CRC, 2006).

Book   MATH   Google Scholar  

Borg, I., Groenen, P. J. F. & Mair, P. MDS Algorithms. In Applied Multidimensional Scaling (eds. Borg, I. et al .) 81–86 (Springer, 2013).

Hamers, L. & Others, A. Similarity measures in scientometric research: The Jaccard index versus Salton’s cosine formula. Inf. Process. Manag. 25 , 315–318 (1989).

Wagner, C. S. & Leydesdorff, L. Network structure, self-organization, and the growth of international collaboration in science. Res. Policy 34 , 1608–1618 (2005).

Hobro, N., Weinman, J. & Hankins, M. Using the self-regulatory model to cluster chronic pain patients: The first step towards identifying relevant treatments?. Pain 108 , 276–283 (2004).

Matura, L. A., McDonough, A. & Carroll, D. L. Cluster analysis of symptoms in pulmonary arterial hypertension: A pilot study. Eur. J. Cardiovasc. Nurs. 11 , 51–61 (2012).

Clatworthy, J., Buick, D., Hankins, M., Weinman, J. & Horne, R. The use and reporting of cluster analysis in health psychology: A review. Br. J. Health Psychol. 10 , 329–358 (2005).

Yadav, J. & Sharma, M. A review of k-mean algorithm. Int. J. Eng. Trends Technol. 4 , 2972–2976 (2013).

Omran, M. G. H., Engelbrecht, A. P. & Salman, A. An overview of clustering methods. Intell. Data Anal. 11 , 583–605 (2007).

Kaur, N. K., Kaur, U. & Singh, U. K-Medoid clustering algorithm-a review. Int. J. Comput. Appl. Technol. 1 , 42–45 (2014).

Maulik, U. & Bandyopadhyay, S. Performance evaluation of some clustering algorithms and validity indices. IEEE Trans. Pattern Anal. Mach. Intell. 24 , 1650–1654 (2002).

Li, P., Burgess, C. & Lund, K. The acquisition of word meaning through global lexical co-occurrences. In Proceedings of the Thirtieth Annual Child (2000).

Recchia, G. & Louwerse, M. M. Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance. Q. J. Exp. Psychol. 68 , 1584–1598 (2015).

Veronese, G., Pepe, A. & Afana, A. Conceptualizing the well-being of helpers living and working in war-like conditions: A mixed-method approach. Int. Soc. Work 59 , 938–952 (2016).

Veronese, G., Pepe, A. & Vigliaroni, M. An exploratory multi-site mixed-method study with migrants at Niger transit centers: The push factors underpinning outward and return migration. Int. Soc. Work 64 , 539–555 (2021).

Richards, B. Type/Token Ratios: What do they really tell us?*. J. Child. Lang. 14 , 201–209 (1987).

Pollack, W. S. ‘I’ness and ‘We’ness: Parallel Lines of Development (Boston University Graduate School, 1982).

Farina, E., Ornaghi, V., Pepe, A., Fiorilli, C. & Grazzani, I. High school student burnout: Is empathy a protective or risk factor?. Front. Psychol. 11 , 897 (2020).

Farina, E., Pepe, A., Ornaghi, V. & Cavioni, V. Trait emotional intelligence and school burnout discriminate between high and low alexithymic profiles: A study with female adolescents. Front. Psychol. 12 , 645215 (2021).

Inchley, J. C., Stevens, G. W. J. M., Samdal, O. & Currie, D. B. Enhancing understanding of adolescent health and well-being: The health behaviour in school-aged children study. J. Adolesc. Health 66 , S3–S5 (2020).

Elmer, T., Mepham, K. & Stadtfeld, C. Students under lockdown: Comparisons of students’ social networks and mental health before and during the COVID-19 crisis in Switzerland. PLoS ONE 15 , e0236337 (2020).

Strittmatter, E. et al. Pathological Internet use among adolescents: Comparing gamers and non-gamers. Psychiatry Res. 228 , 128–135 (2015).

Pieh, C., Plener, P. L., Probst, T., Dale, R. & Humer, E. Mental health in adolescents during COVID-19-related social distancing and home-schooling. SSRN Electron. J. https://doi.org/10.2139/ssrn.3795639 (2021).

Casey, B. J. et al. The storm and stress of adolescence: Insights from human imaging and mouse genetics. Dev. Psychobiol. 52 , 225–235 (2010).

Laursen, B. & Hartl, A. C. Understanding loneliness during adolescence: Developmental changes that increase the risk of perceived social isolation. J. Adolesc. 36 , 1261–1268 (2013).

Singer, M., Bulled, N., Ostrach, B. & Mendenhall, E. Syndemics and the biosocial conception of health. Lancet 389 , 941–950 (2017).

Chervonsky, E. & Hunt, C. Emotion regulation, mental health, and social wellbeing in a young adolescent sample: A concurrent and longitudinal investigation. Emotion 19 , 270–282 (2019).

Jones, N. et al. Compounding inequalities: Adolescent psychosocial wellbeing and resilience among refugee and host communities in Jordan during the COVID-19 pandemic. PLoS ONE 17 , e0261773 (2022).

Download references

Author information

Authors and affiliations.

“R.Massa” Department of Human Sciences for Education, University of Milano-Bicocca, Milan, Italy

Alessandro Pepe & Eleonora Farina

LAB300, University of Milano-Bicocca, Milan, Italy

You can also search for this author in PubMed   Google Scholar

Contributions

A.P. and E.F. conceived the research, wrote the first draft of the article and contributed equally to this work.

Corresponding author

Correspondence to Alessandro Pepe .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Pepe, A., Farina, E. A mixed-method study on adolescents’ well-being during the COVID-19 syndemic emergency. Sci Rep 13 , 871 (2023). https://doi.org/10.1038/s41598-022-24007-w

Download citation

Received : 07 August 2022

Accepted : 08 November 2022

Published : 17 January 2023

DOI : https://doi.org/10.1038/s41598-022-24007-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

The Use of Mixed Methods in Research

• Reference work entry
• First Online: 13 January 2019
• Cite this reference work entry

• Kate A. McBride 2 ,
• Freya MacMillan 3 ,
• Emma S. George 4 &
• Genevieve Z. Steiner 5  

2523 Accesses

9 Citations

Mixed methods research is becoming increasingly popular and is widely acknowledged as a means of achieving a more complex understanding of research problems. Combining both the in-depth, contextual views of qualitative research with the broader generalizations of larger population quantitative approaches, mixed methods research can be used to produce a rigorous and credible source of data. Using this methodology, the same core issue is investigated through the collection, analysis, and interpretation of both types of data within one study or a series of studies. Multiple designs are possible and can be guided by philosophical assumptions. Both qualitative and quantitative data can be collected simultaneously or sequentially (in any order) through a multiphase project. Integration of the two data sources then occurs with consideration is given to the weighting of both sources; these can either be equal or one can be prioritized over the other. Designed as a guide for novice mixed methods researchers, this chapter gives an overview of the historical and philosophical roots of mixed methods research. We also provide a practical overview of its application in health research as well as pragmatic considerations for those wishing to undertake mixed methods research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Appleton JV, King L. Journeying from the philosophical contemplation of constructivism to the methodological pragmatics of health services research. J Adv Nurs. 2002;40(6):641–8.

Article   Google Scholar  

Baba CT, Oliveira IM, Silva AEF, Vieira LM, Cerri NC, Florindo AA, de Oliveira Gomes GA. Evaluating the impact of a walking program in a disadvantaged area: using the RE-AIM framework by mixed methods. BMC Public Health. 2017;17(1):709. https://doi.org/10.1186/s12889-017-4698-5 .

Bryman A. The research question in social research: what is its role? Int J Soc Res Methodol. 2007;10(1):5–20. https://doi.org/10.1080/13645570600655282 .

Bryman A. The end of the paradigm wars. In: Alasuutari P, Bickman L, Brannen J, editors. The Sage handbook of social research methods. Thousand Oaks: Sage; 2008. p. 13–25.

Google Scholar  

Caracelli VJ, Greene JC. Data-analysis strategies for mixed-method evaluation designs. Educ Eval Policy Anal. 1993;15(2):195–207. https://doi.org/10.3102/01623737015002195 .

Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mixed Methods Res. 2010;4(4):342–60. https://doi.org/10.1177/1558689810382916 .

Center for Innovation in Teaching in Research. Choosing a mixed methods design. 2017. Retrieved 15 Nov 2017 from https://cirt.gcu.edu/research/developmentresources/research_ready/mixed_methods/choosing_design .

Creswell JW, Plano Clark VL. Designing and conducting mixed methods research. 3rd ed. Thousand Oaks: Sage; 2018.

Curry LA, Krumholz HM, O’Cathain A, Clark VLP, Cherlin E, Bradley EH. Mixed methods in biomedical and health services research. Circ-Cardiovasc Qual Outcomes. 2013;6(1):119–23. https://doi.org/10.1161/circoutcomes.112.967885 .

Denscombe M. Communities of practice: a research paradigm for the mixed methods approach. J Mixed Methods Res. 2008;2(3):270–83.

Doyle L, Brady A-M, Byrne G. An overview of mixed methods research. J Res Nurs. 2009;14(2):175–85.

Foster NE, Bishop A, Bartlam B, Ogollah R, Barlas P, Holden M, … Young J. Evaluating acupuncture and standard carE for pregnant women with back pain (EASE back): a feasibility study and pilot randomised trial. Health Technol Assess. 2016;20(33):1–236. https://doi.org/10.3310/hta20330 .

Haider AH, Schneider EB, Kodadek LM, Adler RR, Ranjit A, Torain M, … Lau BD. Emergency department query for patient-centered approaches to sexual orientation and gender identity the EQUALITY study. JAMA Intern Med. 2017;177(6):819–828. https://doi.org/10.1001/jamainternmed.2017.0906 .

Harding KE, Taylor NF, Bowers B, Stafford M, Leggat SG. Clinician and patient perspectives of a new model of triage in a community rehabilitation program that reduced waiting time: a qualitative analysis. Aust Health Rev. 2013;37(3):324–30. https://doi.org/10.1071/ah13033 .

Hoddinott P, Britten J, Prescott GJ, Tappin D, Ludbrook A, Godden DJ. Effectiveness of policy to provide breastfeeding groups ( BIG) for pregnant and breastfeeding mothers in primary care: cluster randomised controlled trial. Br Med J. 2009;338:a3026, 10. https://doi.org/10.1136/bmj.a3026 .

Ivankova NV, Creswell JW, Stick SL. Using mixed-methods sequential explanatory design: from theory to practice. Field Methods. 2006;18(1):3–20. https://doi.org/10.1177/1525822x05282260 .

Johnson B, Gray R. A history of philosophical and theoretical issues for mixed methods research. In: Tashakkori A, Teddlie C, editors. Sage handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks: Sage; 2010. p. 69–94.

Chapter   Google Scholar  

Johnson BR, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educ Res. 2004;33(7):14–26. https://doi.org/10.3102/0013189x033007014 .

Keeney S, McKenna H, Fleming P, McIlfatrick S. Attitudes to cancer and cancer prevention: what do people aged 35–54 years think? Eur J Cancer Care. 2010;19(6):769–77. https://doi.org/10.1111/j.1365-2354.2009.01137.x .

McBride KA, Ballinger ML, Schlub TE, Young MA, Tattersall MHN, Kirk J, et al. Psychosocial morbidity in TP53 mutation carriers: is whole-body cancer screening beneficial? Familial Cancer. 2017;16(3):423–32. https://doi.org/10.1007/s10689-016-9964-7 .

Mertens DM. Transformative mixed methods: addressing inequities. Am Behav Sci. 2012;56(6):802–13. https://doi.org/10.1177/0002764211433797 .

Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research – what happens when mixed method findings conflict? BMC Health Serv Res. 2006;6:28. https://doi.org/10.1186/1472-6963-6-28 .

Morse JM. Approaches to qualitative-quantitative methodological triangulation. Nurs Res. 1991a;40(2):120–3.

Morse JM. Principles of mixed methods and multimethod research design. In: Tashakkori A, Teddilie C, editors. SAGE handbook of mixed methods in social and behavioral research. Thousand Oaks: Sage; 1991b.

Mutrie N, Doolin O, Fitzsimons CF, Grant PM, Granat M, Grealy M, et al. Increasing older adults’ walking through primary care: results of a pilot randomized controlled trial. Fam Pract. 2012;29(6):633–42. https://doi.org/10.1093/fampra/cms038 .

Newman I, Ridenour C, Newman C, De Marco G. A typology of research purposes and its relationship to mixed methods in social and behavioral research. Thousand Oaks: Sage; 2003. p. 167–88.

O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8. https://doi.org/10.1258/jhsrp.2007.007074 .

Onwuegbuzie AJ, Collins KMT. A typology of mixed methods sampling designs in social science research. Qual Rep. 2007;12(2):281–316.

Plano Clark VL, Badiee M. Research questions in mixed methods research. In: Tashakkori A, Teddlie C, editors. SAGE handbook of mixed methods in social and behavioral research. 2nd ed. Thousand Oaks: Sage; 2010. p. 275–304.

Prades, J., Algara, M., Espinas, J. A., Farrus, B., Arenas, M., Reyes, V., . . . Borras, J. M. (2017). Understanding variations in the use of hypofractionated radiotherapy and its specific indications for breast cancer: a mixed-methods study. Radiother Oncol, 123(1), 22–28. doi: https://doi.org/10.1016/j.radonc.2017.01.014 .

Tariq S, Woodman J. Using mixed methods in health research. JRSM Short Rep. 2013; 4 (6):1–8. https://doi.org/10.1177/2042533313479197 .

Tashakkori A, Creswell JW. Editorial: the new era of mixed methods. J Mixed Methods Res. 2007;1(1):3–7. https://doi.org/10.1177/2345678906293042 .

Tashakkori A, Teddlie C. The past and future of mixed methods research: from data triangulation to mixed model designs. In: Tashakkori A, Teddlie C, editors. Handbook of mixed methods in social & behavioral research. Thousand Oaks: Sage; 2003. p. 671–701.

Teddlie C, Tashakkori A. Foundations of mixed methods research: integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks: Sage; 2009.

Teddlie C, Yu F. Mixed methods sampling a typology with examples. J Mixed Methods Res. 2007;1(1):77–100. https://doi.org/10.1177/2345678906292430 .

Wellard SJ, Rasmussen B, Savage S, Dunning T. Exploring staff diabetes medication knowledge and practices in regional residential care: triangulation study. J Clin Nurs. 2013;22(13–14):1933–40. https://doi.org/10.1111/jocn.12043 .

Zhang W. Mixed methods application in health intervention research: a multiple case study. Int J Mult Res Approaches. 2014;8(1):24–35. https://doi.org/10.5172/mra.2014.8.1.24 .

Download references

Author information

Authors and affiliations.

School of Medicine and Translational Health Research Institute, Western Sydney University, Sydney, NSW, Australia

Kate A. McBride

School of Science and Health and Translational Health Research Institute (THRI), Western Sydney University, Penrith, NSW, Australia

Freya MacMillan

School of Science and Health, Western Sydney University, Sydney, NSW, Australia

Emma S. George

NICM and Translational Health Research Institute (THRI), Western Sydney University, Penrith, NSW, Australia

Genevieve Z. Steiner

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Kate A. McBride .

Editor information

Editors and affiliations.

School of Science and Health, Western Sydney University, Penrith, NSW, Australia

Pranee Liamputtong

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this entry

Cite this entry.

McBride, K.A., MacMillan, F., George, E.S., Steiner, G.Z. (2019). The Use of Mixed Methods in Research. In: Liamputtong, P. (eds) Handbook of Research Methods in Health Social Sciences. Springer, Singapore. https://doi.org/10.1007/978-981-10-5251-4_97

Download citation

DOI : https://doi.org/10.1007/978-981-10-5251-4_97

Published : 13 January 2019

Publisher Name : Springer, Singapore

Print ISBN : 978-981-10-5250-7

Online ISBN : 978-981-10-5251-4

eBook Packages : Social Sciences Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• News & Highlights

• Publications and Documents
• Postgraduate Education
• Browse Our Courses
• C/T Research Academy
• K12 Investigator Training
• Translational Innovator
• SMART IRB Reliance Request
• Biostatistics Consulting
• Regulatory Support
• Pilot Funding
• Informatics Program
• Community Engagement
• Diversity Inclusion
• Research Enrollment and Diversity
• Harvard Catalyst Profiles

Community Engagement Program

Supporting bi-directional community engagement to improve the relevance, quality, and impact of research.

• Getting Started
• Resources for Equity in Research
• Community-Engaged Student Practice Placement
• Maternal Health Equity
• Youth Mental Health
• Leadership and Membership
• Past Members
• Study Review Rubric
• Community Ambassador Initiative
• Implementation Science Working Group
• Past Webinars & Podcasts
• Policy Atlas
• Community Advisory Board

For more information:

Mixed methods research.

According to the National Institutes of Health , mixed methods strategically integrates or combines rigorous quantitative and qualitative research methods to draw on the strengths of each. Mixed method approaches allow researchers to use a diversity of methods, combining inductive and deductive thinking, and offsetting limitations of exclusively quantitative and qualitative research through a complementary approach that maximizes strengths of each data type and facilitates a more comprehensive understanding of health issues and potential resolutions.¹ Mixed methods may be employed to produce a robust description and interpretation of the data, make quantitative results more understandable, or understand broader applicability of small-sample qualitative findings.

Integration

This refers to the ways in which qualitative and quantitative research activities are brought together to achieve greater insight. Mixed methods is not simply having quantitative and qualitative data available or analyzing and presenting data findings separately. The integration process can occur during data collection, analysis, or in the presentation of results.

¹ NIH Office of Behavioral and Social Sciences Research: Best Practices for Mixed Methods Research in the Health Sciences

Basic Mixed Methods Research Designs 

View image description .

Five Key Questions for Getting Started

• What do you want to know?
• What will be the detailed quantitative, qualitative, and mixed methods research questions that you hope to address?
• What quantitative and qualitative data will you collect and analyze?
• Which rigorous methods will you use to collect data and/or engage stakeholders?
• How will you integrate the data in a way that allows you to address the first question?

Rationale for Using Mixed Methods

• Obtain different, multiple perspectives: validation
• Build comprehensive understanding
• Explain statistical results in more depth
• Have better contextualized measures
• Track the process of program or intervention
• Study patient-centered outcomes and stakeholder engagement

Sample Mixed Methods Research Study

The EQUALITY study used an exploratory sequential design to identify the optimal patient-centered approach to collect sexual orientation data in the emergency department.

Qualitative Data Collection and Analysis : Semi-structured interviews with patients of different sexual orientation, age, race/ethnicity, as well as healthcare professionals of different roles, age, and race/ethnicity.

Builds Into : Themes identified in the interviews were used to develop questions for the national survey.

Quantitative Data Collection and Analysis : Representative national survey of patients and healthcare professionals on the topic of reporting gender identity and sexual orientation in healthcare.

Other Resources:

  Introduction to Mixed Methods Research : Harvard Catalyst’s eight-week online course offers an opportunity for investigators who want to understand and apply a mixed methods approach to their research.

Best Practices for Mixed Methods Research in the Health Sciences [PDF] : This guide provides a detailed overview of mixed methods designs, best practices, and application to various types of grants and projects.

Mixed Methods Research Training Program for the Health Sciences (MMRTP ): Selected scholars for this summer training program, hosted by Johns Hopkins’ Bloomberg School of Public Health, have access to webinars, resources, a retreat to discuss their research project with expert faculty, and are matched with mixed methods consultants for ongoing support.

Michigan Mixed Methods : University of Michigan Mixed Methods program offers a variety of resources, including short web videos and recommended reading.

To use a mixed methods approach, you may want to first brush up on your qualitative skills. Below are a few helpful resources specific to qualitative research:

• Qualitative Research Guidelines Project : A comprehensive guide for designing, writing, reviewing and reporting qualitative research.
• Fundamentals of Qualitative Research Methods – What is Qualitative Research : A six-module web video series covering essential topics in qualitative research, including what is qualitative research and how to use the most common methods, in-depth interviews, and focus groups.

View PDF of the above information.

• What is mixed methods research?

Last updated

20 February 2023

Reviewed by

Miroslav Damyanov

By blending both quantitative and qualitative data, mixed methods research allows for a more thorough exploration of a research question. It can answer complex research queries that cannot be solved with either qualitative or quantitative research .

Analyze your mixed methods research

Dovetail streamlines analysis to help you uncover and share actionable insights

Mixed methods research combines the elements of two types of research: quantitative and qualitative.

Quantitative data is collected through the use of surveys and experiments, for example, containing numerical measures such as ages, scores, and percentages. 

Qualitative data involves non-numerical measures like beliefs, motivations, attitudes, and experiences, often derived through interviews and focus group research to gain a deeper understanding of a research question or phenomenon.

Mixed methods research is often used in the behavioral, health, and social sciences, as it allows for the collection of numerical and non-numerical data.

• When to use mixed methods research

Mixed methods research is a great choice when quantitative or qualitative data alone will not sufficiently answer a research question. By collecting and analyzing both quantitative and qualitative data in the same study, you can draw more meaningful conclusions. 

There are several reasons why mixed methods research can be beneficial, including generalizability, contextualization, and credibility. 

For example, let's say you are conducting a survey about consumer preferences for a certain product. You could collect only quantitative data, such as how many people prefer each product and their demographics. Or you could supplement your quantitative data with qualitative data, such as interviews and focus groups , to get a better sense of why people prefer one product over another.

It is important to note that mixed methods research does not only mean collecting both types of data. Rather, it also requires carefully considering the relationship between the two and method flexibility.

You may find differing or even conflicting results by combining quantitative and qualitative data . It is up to the researcher to then carefully analyze the results and consider them in the context of the research question to draw meaningful conclusions.

When designing a mixed methods study, it is important to consider your research approach, research questions, and available data. Think about how you can use different techniques to integrate the data to provide an answer to your research question.

• Mixed methods research design

A mixed methods research design  is   an approach to collecting and analyzing both qualitative and quantitative data in a single study.

Mixed methods designs allow for method flexibility and can provide differing and even conflicting results. Examples of mixed methods research designs include convergent parallel, explanatory sequential, and exploratory sequential.

By integrating data from both quantitative and qualitative sources, researchers can gain valuable insights into their research topic . For example, a study looking into the impact of technology on learning could use surveys to measure quantitative data on students' use of technology in the classroom. At the same time, interviews or focus groups can provide qualitative data on students' experiences and opinions.

• Types of mixed method research designs

Researchers often struggle to put mixed methods research into practice, as it is challenging and can lead to research bias. Although mixed methods research can reveal differences or conflicting results between studies, it can also offer method flexibility.

Designing a mixed methods study can be broken down into four types: convergent parallel, embedded, explanatory sequential, and exploratory sequential.

Convergent parallel

The convergent parallel design is when data collection and analysis of both quantitative and qualitative data occur simultaneously and are analyzed separately. This design aims to create mutually exclusive sets of data that inform each other. 

For example, you might interview people who live in a certain neighborhood while also conducting a survey of the same people to determine their satisfaction with the area.

Embedded design

The embedded design is when the quantitative and qualitative data are collected simultaneously, but the qualitative data is embedded within the quantitative data. This design is best used when you want to focus on the quantitative data but still need to understand how the qualitative data further explains it.

For instance, you may survey students about their opinions of an online learning platform and conduct individual interviews to gain further insight into their responses.

Explanatory sequential design

In an explanatory sequential design, quantitative data is collected first, followed by qualitative data. This design is used when you want to further explain a set of quantitative data with additional qualitative information.

An example of this would be if you surveyed employees at a company about their satisfaction with their job and then conducted interviews to gain more information about why they responded the way they did.

Exploratory sequential design

The exploratory sequential design collects qualitative data first, followed by quantitative data. This type of mixed methods research is used when the goal is to explore a topic before collecting any quantitative data.

An example of this could be studying how parents interact with their children by conducting interviews and then using a survey to further explore and measure these interactions.

Integrating data in mixed methods studies can be challenging, but it can be done successfully with careful planning.

No matter which type of design you choose, understanding and applying these principles can help you draw meaningful conclusions from your research.

• Strengths of mixed methods research

Mixed methods research designs combine the strengths of qualitative and quantitative data, deepening and enriching qualitative results with quantitative data and validating quantitative findings with qualitative data. This method offers more flexibility in designing research, combining theory generation and hypothesis testing, and being less tied to disciplines and established research paradigms.

Take the example of a study examining the impact of exercise on mental health. Mixed methods research would allow for a comprehensive look at the issue from different angles. 

Researchers could begin by collecting quantitative data through surveys to get an overall view of the participants' levels of physical activity and mental health. Qualitative interviews would follow this to explore the underlying dynamics of participants' experiences of exercise, physical activity, and mental health in greater detail.

Through a mixed methods approach, researchers could more easily compare and contrast their results to better understand the phenomenon as a whole.  

Additionally, mixed methods research is useful when there are conflicting or differing results in different studies. By combining both quantitative and qualitative data, mixed methods research can offer insights into why those differences exist.

For example, if a quantitative survey yields one result while a qualitative interview yields another, mixed methods research can help identify what factors influence these differences by integrating data from both sources.

Overall, mixed methods research designs offer a range of advantages for studying complex phenomena. They can provide insight into different elements of a phenomenon in ways that are not possible with either qualitative or quantitative data alone. Additionally, they allow researchers to integrate data from multiple sources to gain a deeper understanding of the phenomenon in question.  

• Challenges of mixed methods research

Mixed methods research is labor-intensive and often requires interdisciplinary teams of researchers to collaborate. It also has the potential to cost more than conducting a stand alone qualitative or quantitative study . 

Interpreting the results of mixed methods research can be tricky, as it can involve conflicting or differing results. Researchers must find ways to systematically compare the results from different sources and methods to avoid bias.

For example, imagine a situation where a team of researchers has employed an explanatory sequential design for their mixed methods study. After collecting data from both the quantitative and qualitative stages, the team finds that the two sets of data provide differing results. This could be challenging for the team, as they must now decide how to effectively integrate the two types of data in order to reach meaningful conclusions. The team would need to identify method flexibility and be strategic when integrating data in order to draw meaningful conclusions from the conflicting results.

• Advanced frameworks in mixed methods research

Mixed methods research offers powerful tools for investigating complex processes and systems, such as in health and healthcare.

Besides the three basic mixed method designs—exploratory sequential, explanatory sequential, and convergent parallel—you can use one of the four advanced frameworks to extend mixed methods research designs. These include multistage, intervention, case study , and participatory. 

This framework mixes qualitative and quantitative data collection methods in stages to gather a more nuanced view of the research question. An example of this is a study that first has an online survey to collect initial data and is followed by in-depth interviews to gain further insights.

Intervention

This design involves collecting quantitative data and then taking action, usually in the form of an intervention or intervention program. An example of this could be a research team who collects data from a group of participants, evaluates it, and then implements an intervention program based on their findings .

This utilizes both qualitative and quantitative research methods to analyze a single case. The researcher will examine the specific case in detail to understand the factors influencing it. An example of this could be a study of a specific business organization to understand the organizational dynamics and culture within the organization.

Participatory

This type of research focuses on the involvement of participants in the research process. It involves the active participation of participants in formulating and developing research questions, data collection, and analysis.

An example of this could be a study that involves forming focus groups with participants who actively develop the research questions and then provide feedback during the data collection and analysis stages.

The flexibility of mixed methods research designs means that researchers can choose any combination of the four frameworks outlined above and other methodologies , such as convergent parallel, explanatory sequential, and exploratory sequential, to suit their particular needs.

Through this method's flexibility, researchers can gain multiple perspectives and uncover differing or even conflicting results when integrating data.

When it comes to integration at the methods level, there are four approaches.

Connecting involves collecting both qualitative and quantitative data during different phases of the research.

Building involves the collection of both quantitative and qualitative data within a single phase.

Merging involves the concurrent collection of both qualitative and quantitative data.

Embedding involves including qualitative data within a quantitative study or vice versa.

• Techniques for integrating data in mixed method studies

Integrating data is an important step in mixed methods research designs. It allows researchers to gain further understanding from their research and gives credibility to the integration process. There are three main techniques for integrating data in mixed methods studies: triangulation protocol, following a thread, and the mixed methods matrix.

Triangulation protocol

This integration method combines different methods with differing or conflicting results to generate one unified answer.

For example, if a researcher wanted to know what type of music teenagers enjoy listening to, they might employ a survey of 1,000 teenagers as well as five focus group interviews to investigate this. The results might differ; the survey may find that rap is the most popular genre, whereas the focus groups may suggest rock music is more widely listened to. 

The researcher can then use the triangulation protocol to come up with a unified answer—such as that both rap and rock music are popular genres for teenage listeners. 

Following a thread

This is another method of integration where the researcher follows the same theme or idea from one method of data collection to the next. 

A research design that follows a thread starts by collecting quantitative data on a specific issue, followed by collecting qualitative data to explain the results. This allows whoever is conducting the research to detect any conflicting information and further look into the conflicting information to understand what is really going on.

For example, a researcher who used this research method might collect quantitative data about how satisfied employees are with their jobs at a certain company, followed by qualitative interviews to investigate why job satisfaction levels are low. They could then use the results to explore any conflicting or differing results, allowing them to gain a deeper understanding of job satisfaction at the company. 

By following a thread, the researcher can explore various research topics related to the original issue and gain a more comprehensive view of the issue.

Mixed methods matrix

This technique is a visual representation of the different types of mixed methods research designs and the order in which they should be implemented. It enables researchers to quickly assess their research design and adjust it as needed. 

The matrix consists of four boxes with four different types of mixed methods research designs: convergent parallel, explanatory sequential, exploratory sequential, and method flexibility. 

For example, imagine a researcher who wanted to understand why people don't exercise regularly. To answer this question, they could use a convergent parallel design, collecting both quantitative (e.g., survey responses) and qualitative (e.g., interviews) data simultaneously.

If the researcher found conflicting results, they could switch to an explanatory sequential design and collect quantitative data first, then follow up with qualitative data if needed. This way, the researcher can make adjustments based on their findings and integrate their data more effectively.

Mixed methods research is a powerful tool for understanding complex research topics. Using qualitative and quantitative data in one study allows researchers to understand their subject more deeply. 

Mixed methods research designs such as convergent parallel, explanatory sequential, and exploratory sequential provide method flexibility, enabling researchers to collect both types of data while avoiding the limitations of either approach alone.

However, it's important to remember that mixed methods research can produce differing or even conflicting results, so it's important to be aware of the potential pitfalls and take steps to ensure that data is being correctly integrated. If used effectively, mixed methods research can offer valuable insight into topics that would otherwise remain largely unexplored.

What is an example of mixed methods research?

An example of mixed methods research is a study that combines quantitative and qualitative data. This type of research uses surveys, interviews, and observations to collect data from multiple sources.

Which sampling method is best for mixed methods?

It depends on the research objectives, but a few methods are often used in mixed methods research designs. These include snowball sampling, convenience sampling, and purposive sampling. Each method has its own advantages and disadvantages.

What is the difference between mixed methods and multiple methods?

Mixed methods research combines quantitative and qualitative data in a single study. Multiple methods involve collecting data from different sources, such as surveys and interviews, but not necessarily combining them into one analysis. Mixed methods offer greater flexibility but can lead to differing or conflicting results when integrating data.

Get started today

Go from raw data to valuable insights with a flexible research platform

Editor’s picks

Last updated: 21 December 2023

Last updated: 16 December 2023

Last updated: 6 October 2023

Last updated: 25 November 2023

Last updated: 12 May 2023

Last updated: 15 February 2024

Last updated: 11 March 2024

Last updated: 12 December 2023

Last updated: 18 May 2023

Last updated: 6 March 2024

Last updated: 10 April 2023

Last updated: 20 December 2023

Latest articles

Related topics, log in or sign up.

Get started for free

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Volume 20, Issue 3
• Mixed methods research: expanding the evidence base
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Allison Shorten 1 ,
• Joanna Smith 2
• 1 School of Nursing , University of Alabama at Birmingham , USA
• 2 Children's Nursing, School of Healthcare , University of Leeds , UK
• Correspondence to Dr Allison Shorten, School of Nursing, University of Alabama at Birmingham, 1720 2nd Ave South, Birmingham, AL, 35294, USA; [email protected]; ashorten{at}uab.edu

https://doi.org/10.1136/eb-2017-102699

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

‘Mixed methods’ is a research approach whereby researchers collect and analyse both quantitative and qualitative data within the same study. 1 2 Growth of mixed methods research in nursing and healthcare has occurred at a time of internationally increasing complexity in healthcare delivery. Mixed methods research draws on potential strengths of both qualitative and quantitative methods, 3 allowing researchers to explore diverse perspectives and uncover relationships that exist between the intricate layers of our multifaceted research questions. As providers and policy makers strive to ensure quality and safety for patients and families, researchers can use mixed methods to explore contemporary healthcare trends and practices across increasingly diverse practice settings.

What is mixed methods research?

Mixed methods research requires a purposeful mixing of methods in data collection, data analysis and interpretation of the evidence. The key word is ‘mixed’, as an essential step in the mixed methods approach is data linkage, or integration at an appropriate stage in the research process. 4 Purposeful data integration enables researchers to seek a more panoramic view of their research landscape, viewing phenomena from different viewpoints and through diverse research lenses. For example, in a randomised controlled trial (RCT) evaluating a decision aid for women making choices about birth after caesarean, quantitative data were collected to assess knowledge change, levels of decisional conflict, birth choices and outcomes. 5 Qualitative narrative data were collected to gain insight into women’s decision-making experiences and factors that influenced their choices for mode of birth. 5

In contrast, multimethod research uses a single research paradigm, either quantitative or qualitative. Data are collected and analysed using different methods within the same paradigm. 6 7 For example, in a multimethods qualitative study investigating parent–professional shared decision-making regarding diagnosis of suspected shunt malfunction in children, data collection included audio recordings of admission consultations and interviews 1 week post consultation, with interactions analysed using conversational analysis and the framework approach for the interview data. 8

What are the strengths and challenges in using mixed methods?

Selecting the right research method starts with identifying the research question and study aims. A mixed methods design is appropriate for answering research questions that neither quantitative nor qualitative methods could answer alone. 4 9–11 Mixed methods can be used to gain a better understanding of connections or contradictions between qualitative and quantitative data; they can provide opportunities for participants to have a strong voice and share their experiences across the research process, and they can facilitate different avenues of exploration that enrich the evidence and enable questions to be answered more deeply. 11 Mixed methods can facilitate greater scholarly interaction and enrich the experiences of researchers as different perspectives illuminate the issues being studied. 11

The process of mixing methods within one study, however, can add to the complexity of conducting research. It often requires more resources (time and personnel) and additional research training, as multidisciplinary research teams need to become conversant with alternative research paradigms and different approaches to sample selection, data collection, data analysis and data synthesis or integration. 11

What are the different types of mixed methods designs?

Mixed methods research comprises different types of design categories, including explanatory, exploratory, parallel and nested (embedded) designs. 2   Table 1 summarises the characteristics of each design, the process used and models of connecting or integrating data. For each type of research, an example was created to illustrate how each study design might be applied to address similar but different nursing research aims within the same general nursing research area.

• View inline

Types of mixed methods designs*

What should be considered when evaluating mixed methods research?

When reading mixed methods research or writing a proposal using mixed methods to answer a research question, the six questions below are a useful guide 12 :

Does the research question justify the use of mixed methods?

Is the method sequence clearly described, logical in flow and well aligned with study aims?

Is data collection and analysis clearly described and well aligned with study aims?

Does one method dominate the other or are they equally important?

Did the use of one method limit or confound the other method?

When, how and by whom is data integration (mixing) achieved?

For more detail of the evaluation guide, refer to the McMaster University Mixed Methods Appraisal Tool. 12 The quality checklist for appraising published mixed methods research could also be used as a design checklist when planning mixed methods studies.

• Elliot AE , et al
• Creswell JW ,
• Plano ClarkV L
• Greene JC ,
• Caracelli VJ ,
• Ivankova NV
• Shorten A ,
• Shorten B ,
• Halcomb E ,
• Cheater F ,
• Bekker H , et al
• Tashakkori A ,
• Creswell JW
• 12. ↵ National Collaborating Centre for Methods and Tools . Appraising qualitative, quantitative, and mixed methods studies included in mixed studies reviews: the MMAT . Hamilton, ON : BMJ Publishing Group , 2015 . http://www.nccmt.ca/resources/search/232 (accessed May 2017) .

Competing interests None declared.

Provenance and peer review Commissioned; internally peer reviewed.

Read the full text or download the PDF:

• Research article
• Open access
• Published: 24 September 2018

A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults

• Gabrielle Lindsay-Smith   ORCID: orcid.org/0000-0003-3864-1412 1 ,
• Grant O’Sullivan 1 ,
• Rochelle Eime 1 , 2 ,
• Jack Harvey 1 , 2 &
• Jannique G. Z. van Uffelen 1 , 3  

BMC Geriatrics volume  18 , Article number:  226 ( 2018 ) Cite this article

71k Accesses

30 Citations

12 Altmetric

Metrics details

Social wellbeing factors such as loneliness and social support have a major impact on the health of older adults and can contribute to physical and mental wellbeing. However, with increasing age, social contacts and social support typically decrease and levels of loneliness increase. Group social engagement appears to have additional benefits for the health of older adults compared to socialising individually with friends and family, but further research is required to confirm whether group activities can be beneficial for the social wellbeing of older adults.

This one-year longitudinal mixed methods study investigated the effect of joining a community group, offering a range of social and physical activities, on social wellbeing of adults with a mean age of 70. The study combined a quantitative survey assessing loneliness and social support ( n  = 28; three time-points, analysed using linear mixed models) and a qualitative focus group study ( n  = 11, analysed using thematic analysis) of members from Life Activities Clubs Victoria, Australia.

There was a significant reduction in loneliness ( p  = 0.023) and a trend toward an increase in social support ( p  = 0.056) in the first year after joining. The focus group confirmed these observations and suggested that social support may take longer than 1 year to develop. Focus groups also identified that group membership provided important opportunities for developing new and diverse social connections through shared interest and experience. These connections were key in improving the social wellbeing of members, especially in their sense of feeling supported or connected and less lonely. Participants agreed that increasing connections was especially beneficial following significant life events such as retirement, moving to a new house or partners becoming unwell.

Conclusions

Becoming a member of a community group offering social and physical activities may improve social wellbeing in older adults, especially following significant life events such as retirement or moving-house, where social network changes. These results indicate that ageing policy and strategies would benefit from encouraging long-term participation in social groups to assist in adapting to changes that occur in later life and optimise healthy ageing.

Peer Review reports

Ageing population and the need to age well

Between 2015 and 2050 it is predicted that globally the number of adults over the age of 60 will more than double [ 1 ]. Increasing age is associated with a greater risk of chronic illnesses such as cardio vascular disease and cancer [ 2 ] and reduced functional capacity [ 3 , 4 ]. Consequently, an ageing population will continue to place considerable pressure on the health care systems.

However, it is also important to consider the individuals themselves and self-perceived good health is very important for the individual wellbeing and life-satisfaction of older adults [ 5 ]. The terms “successful ageing” [ 6 ] and “healthy ageing” [ 5 ] have been used to define a broader concept of ageing well, which not only includes factors relating to medically defined health but also wellbeing. Unfortunately, there is no agreed definition for what exactly constitutes healthy or successful ageing, with studies using a range of definitions. A review of 28 quantitative studies found that successful ageing was defined differently in each, with the majority only considering measures of disability or physical functioning. Social and wellbeing factors were included in only a few of the studies [ 7 ].

In contrast, qualitative studies of older adults’ opinions on successful ageing have found that while good physical and mental health and maintaining physical activity levels are agreed to assist successful ageing, being independent or doing something of value, acceptance of ageing, life satisfaction, social connectedness or keeping socially active were of greater importance [ 8 , 9 , 10 ].

In light of these findings, the definition that is most inclusive is “healthy ageing” defined by the World Health Organisation as “the process of developing and maintaining the functional ability (defined as a combination of intrinsic capacity and physical and social environmental characteristics), that enables well-being in older age” (p28) [ 5 ].This definition, and those provided in the research of older adults’ perceptions of successful ageing, highlight social engagement and social support as important factors contributing to successful ageing, in addition to being important social determinants of health [ 11 , 12 ].

Social determinants of health, including loneliness and social support, are important predictors of physical, cognitive and mental health and wellbeing in adults [ 12 ] and older adults [ 13 , 14 , 15 ]. Loneliness is defined as a perception of an inadequacy in the quality or quantity of one’s social relationships [ 16 ]. Social support, has various definitions but generally it relates to social relationships that are reciprocal, accessible and reliable and provide any or a combination of supportive resources (e.g. emotional, information, practical) and can be measured as perceived or received support [ 17 ]. These types of social determinants differ from those related to inequality (health gap social determinants) and are sometimes referred to as ‘social cure’ social determinants [ 11 ]. They will be referred to as ‘social wellbeing’ outcome measures in this study.

Unfortunately, with advancing age, there is often diminishing social support, leading to social isolation and loneliness [ 18 , 19 ]. Large nationally representative studies of adults and older adults reported that social activity predicted maintenance or improvement of life satisfaction as well as physical activity levels [ 20 ], however older adults spent less time in social activity than middle age adults.

Social wellbeing and health

A number of longitudinal studies have found that social isolation for older adults is a significant predictor of mortality and institutionalisation [ 21 , 22 , 23 ]. A meta-analysis by Holt-Lunstadt [ 12 ] reported that social determinants of health, including social integration and social support (including loneliness and lack of perceived social support) to be equal to, or a greater risk to mortality as common behavioural risk factors such as smoking, physical inactivity and obesity. Loneliness is independently associated with poor physical and mental health in the general population, and especially in older adults [ 13 , 14 , 15 ]. Adequate perceived social support has also been consistently associated with improved mental and physical health in both general and older adults [ 20 , 24 , 25 , 26 , 27 , 28 , 29 ]. The mechanism suggested for this association is that social support buffers the negative impacts of stressful situations and life events [ 30 ]. The above research demonstrates the benefit of social engagement for older adults; in turn this highlights the importance of strategies that reduce loneliness and improve social support and social connectedness for older adults.

Socialising in groups seems to be especially important for the health and wellbeing of older adults who may be adjusting to significant life events [ 26 , 31 , 32 , 33 ]. This is sometimes referred to as social engagement or social companionship [ 26 , 30 , 31 ]. It seems that the mechanism enabling such health benefits with group participation is through strengthening of social identification, which in turn increases social support [ 31 , 34 , 35 ]. Furthermore, involvement in community groups can be a sustainable strategy to reduce loneliness and increase social support in older adults, as they are generally low cost and run by volunteers [ 36 , 37 , 38 , 39 ].

Despite the demonstrated importance of social factors for successful ageing and the established risk associated with reduced social engagement as people age, few in-depth studies have longitudinally investigated the impact of community groups on social wellbeing. For example, a non-significant increase in social support and reduction in depression was found in a year-long randomised controlled trial conducted in senior centres in Norway with lonely older adults in poor physical and mental health [ 37 ]. Some qualitative studies have reported that community groups and senior centres can contribute to fun and socialisation for older adults, however social wellbeing was not the primary focus of the studies [ 38 , 40 , 41 ]. Given that social wellbeing is a broad and important area for the health and quality of life in older adults, an in-depth study is warranted to understand how it can be maximised in older adults. This mixed methods case study of an existing community aims to: i) examine whether loneliness and social support of new members of Life Activities Clubs (LACs) changes in the year after joining and ii) conduct an in-depth exploration of how social wellbeing changes in new and longer-term members of LACs.

A mixed methods study was chosen as the design for this research to enable an in-depth exploration of how loneliness and social support may change as a result of joining a community group. A case study was conducted using a concurrent mixed-methods design, with a qualitative component giving context to the quantitative results. Where the survey focused on the impact of group membership on social support and loneliness, the focus groups were an open discussion of the benefits in the lived context of LAC membership. The synthesis of the two sections of the study was undertaken at the time of interpretation of the results [ 42 ].

The two parts of our study were as follows:

a longitudinal survey (three time points over 1 year: baseline, 6 and 12 months). This part of the study formed the quantitative results;

a focus group study of members of the same organisation (qualitative).

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]) All participants provided informed consent to partake in the study prior to undertaking the first survey or focus group.

Setting and participants

Life activities clubs victoria.

Life Activities Clubs Victoria (LACVI) is a large not-for-profit group with 23 independently run Life Activities Clubs (LACs) based in both rural and metropolitan Victoria. It has approximately 4000 members. The organisation was established to assist in providing physical, social and recreational activities as well as education and motivational support to older adults managing significant change in their lives, especially retirement.

Eighteen out of 23 LAC clubs agreed to take part in the survey study. During the sampling period from May 2014 to December 2016, new members from the participating clubs were given information about the study and invited to take part. Invitations took place in the form of flyers distributed with new membership material.

Inclusion/ exclusion criteria

Community-dwelling older adults who self-reported that they could walk at least 100 m and who were new members to LACVI and able to complete a survey in English were eligible to participate. New members were defined as people who had never been members of LACVI or who had not been members in the last 2 years.

To ensure that the cohort of participants were of a similar functional level, people with significant health problems limiting them from being able to walk 100 m were excluded from participating in the study.

Once informed consent was received, the participants were invited to complete a self-report survey in either paper or online format (depending on preference). This first survey comprised the baseline data and the same survey was completed 6 months and 12 months after this initial time point. Participants were sent reminders if they had not completed each survey more than 2 weeks after each was delivered and then again 1 week later.

Focus groups

Two focus groups (FGs) were conducted with new and longer-term members of LACs. The first FG ( n  = 6) consisted of members who undertook physical activity in their LAC (e.g. walking groups, tennis, cycling). The second FG ( n  = 5) consisted of members who took part in activities with a non-physical activity (PA) focus (e.g. book groups, social groups, craft or cultural groups). LACs offer both social and physical activities and it was important to the study to capture both types of groups, but they were kept separate to assist participants in feeling a sense of commonality with other members and improving group dynamic and participation in the discussions [ 43 ]. Of the people who participated in the longitudinal survey study, seven also participated in the FGs.

The FG interviews were facilitated by one researcher (GLS) and notes around non-verbal communication, moments of divergence and convergence amongst group members, and other notable items were taken by a second researcher (GOS). Both researchers wrote additional notes after the focus groups and these were used in the analysis of themes. Focus groups were recorded and later transcribed verbatim by a professional transcriptionist, including identification of each participant speaking. One researcher (GLS) reviewed each transcription to check for any errors and made any required modifications before importing the transcriptions into NVivo for analysis. The transcriber identified each focus group participant so themes for individuals or other age or gender specific trends could be identified.

Dependent variables

• Social support

Social support was assessed using the Duke–UNC Functional Social support questionnaire [ 44 ]. This scale specifically measures participant perceived functional social support in two areas; i) confidant support (5 questions; e.g. chances to talk to others) and ii) affective support (3 questions; e.g. people who care about them). Participants rated each component of support on a 5-item likert scale between ‘much less than I would like’ (1 point) to ‘as much as I would like’ (5 points). The total score used for analysis was the mean of the eight scores (low social support = 1, maximum social support = 5). Construct validity, concurrent validity and discriminant validity are acceptable for confidant and affective support items in the survey in the general population [ 44 ].

Loneliness was measured using the de Jong Gierveld and UCLA-3 item loneliness scales developed for use in many populations including older adults [ 45 ]. The 11-item de Jong Gierveld loneliness scale (DJG loneliness) [ 46 ] is a multi-dimensional measure of loneliness and contains five positively worded and six negatively worded items. The items fall into four subscales; feelings of severe loneliness, feelings connected with specific problem situations, missing companionship, feelings of belongingness. The total score is the sum of the items scores (i.e. 11–55): 11 is low loneliness and 55 is severe loneliness. Self-administered versions of this scale have good internal consistency (> = 0.8) and inter-item homogeneity and person scalability that is as good or better than when conducted as face-to face interviews. The validity and reliability for the scale is adequate [ 47 ]. The UCLA 3-item loneliness scale consists of three questions about how often participants feel they lack companionship, feel left out and feel isolated. The responses are given on a three-point scale ranging from hardly ever (1) to often (3). The final score is the sum of these three items with the range being from lowest loneliness (3) to highest loneliness (9). Reliability of the scale is good, (alpha = 0.72) as are discriminant validity and internal consistency [ 48 ]. The scale is commonly used to measure loneliness with older adults ([ 49 ] – review), [ 50 , 51 ].

Sociodemographic variables

The following sociodemographic characteristics were collected in both the survey and the focus groups: age, sex, highest level of education, main life occupation [ 52 ], current employment, ability to manage on income available, present marital status, country of birth, area of residence [ 53 ]. They are categorised as indicated in Table  2 .

Health variables

The following health variables were collected: Self-rated general health (from SF-12) [ 54 ] and Functional health (ability to walk 100 m- formed part of the inclusion criteria) [ 55 ]. See Table 2 for details about the categories of these variables.

The effects of becoming a member on quantitative outcome variables (i.e. Social support, DJG loneliness and UCLA loneliness) were analysed using linear mixed models (LMM). LMM enabled testing for the presence of intra-subject random effects, or equivalently, correlation of subjects’ measures over time (baseline, 6-months and 12 months). Three correlation structures were examined: independence (no correlation), compound symmetry (constant correlation of each subjects’ measures over the three time points) and autoregressive (correlation diminishing with increase in spacing in time). The best fitting correlation structure was compound symmetry; this is equivalent to a random intercept component for each subject. The LMM incorporated longitudinal trends over time, with adjustment for age as a potential confounder. Statistical analyses were conducted using SPSS for windows (v24).

UCLA loneliness and social support residuals were not normally distributed and these scales were Log10 transformed for statistical analysis.

Analyses were all adjusted for age, group attendance (calculated as average attendance at 6 and 12 months) and employment status at baseline (Full-time, Part-time, not working).

Focus group transcripts were analysed using thematic analysis [ 56 , 57 ], a flexible qualitative methodology that can be used with a variety of epistemologies, approaches and analysis methods [ 56 ]. The transcribed data were analysed using a combination of theoretical and inductive thematic analysis [ 56 ]. It was theorised that membership in a LAC would assist with social factors relating to healthy ageing [ 5 ], possibly through a social identity pathway [ 58 ], although we wanted to explore this. Semantic themes were drawn from these codes in order to conduct a pragmatic evaluation of the LACVI programs [ 56 ]. Analytic rigour in the qualitative analysis was ensured through source and analyst triangulation. Transcriptions were compared to notes taken during the focus groups by the researchers (GOS and GLS). In addition, Initial coding and themes (by GLS) were checked by a second researcher (GOS) and any disagreements regarding coding and themes were discussed prior to finalisation of codes and themes [ 57 ].

Sociodemographic and health characteristics of the 28 participants who completed the survey study are reported in Table  1 . The mean age of the participants was 66.9 and 75% were female. These demographics are representative of the entire LACVI membership. Education levels varied, with 21% being university educated, and the remainder completing high school or technical certificates. Two thirds of participants were not married. Some sociodemographic characteristics changed slightly at 6 and 12 months, mainly employment (18% in paid employment at baseline and 11% at 12-months) and ability to manage on income (36% reporting trouble managing on their income at baseline and 46% at 12 months). Almost 90% of the participants described themselves as being in good-excellent health.

Types of activities

There were a variety of types of activities that participants took part in: physical activities such as walking groups ( n  = 7), table tennis ( n  = 5), dancing class ( n  = 2), exercise class ( n  = 1), bowls ( n  = 2), golf ( n  = 3), cycling groups ( n  = 1) and non-physical leisure activities such as art and literature groups ( n  = 5), craft groups ( n  = 5), entertainment groups ( n  = 12), food/dine out groups ( n  = 18) and other sedentary leisure activities (e.g. mah jong, cards),( n  = 4). A number of people took part in more than one activity.

Frequency of attendance at LACVI and changes in social wellbeing

At six and 12 months, participants indicated how many times in the last month they attended different types of activities at their LAC. Most participants maintained the same frequency of participation over both time points. Only four people participated more frequently at 12 than at 6 months and nine reduced participation levels. The latter group included predominantly those who reduced from more than two times per week at 6 months to 2×/week at 6 months to one to two times per week ( n  = 5) or less than one time per week ( n  = 2) at 12 months. Average weekly club attendance at six and 12 months was included as a covariate in the statistical model.

Outcome measures

Overall, participants reported moderate social support and loneliness levels at baseline (See Table 2 ). Loneliness, as measured by both scales, reduced significantly over time. There was a significant effect of time on the DJG loneliness scores (F (2, 52) = 3.83, p  = 0.028), with Post-Hoc analysis indicating a reduction in DJG loneliness between baseline and 12 months ( p  = 0.008). UCLA loneliness scores (transformed variable) also changed significantly over time (F (2, 52) = 4.08, p  = 0.023). Post hoc tests indicated a reduction in UCLA loneliness between baseline and 6 months ( p  = 0.007). There was a small non-significant increase in social support (F (2, 53) =2.88, p  = 0.065) during the first year of membership (see Table 2 and Figs. 1 and 2 ).

DJG loneliness for all participants over first year of membership at LAC club ( n  = 28).

*Represents significant difference compared to baseline ( p  < 0.01)

UCLA loneliness score for all participants over first year of membership at LAC club ( n  = 28).

*Indicates log values of the variable at 6-months were significantly different from baseline ( p  < 0.01)

In total, 11 participants attended the two focus groups, six people who participated in PA clubs (four women) and five who participated in social clubs (all women). All focus group participants were either retired ( n  = 9) or semi-retired ( n  = 2). The mean age of participants was 67 years (see Table 2 for further details). Most of the participants (82%) had been members of a LAC for less than 2 years and two females in the social group had been members of LAC clubs for 5 and 10 years respectively.

Analysis of the focus group transcripts identified two themes relating to social benefits of group participation; i) Social resources and ii) Social wellbeing (see Fig. 3 ). Group discussion suggested that membership of a LAC provides access to more social resources through greater and diverse social contact and opportunity. It is through this improvement in social resources that social wellbeing may improve.

Themes arising from focus group discussion around the benefits of LAC membership

Social resources

The social resources theme referred to an increase in the availability and variety of social connections that resulted from becoming a member of a LAC. The social nature of the groups enabled an expansion and diversification of members’ social network and improved their sense of social connectedness. There was widespread agreement in both the focus groups that significant life events, especially retirement, illness or death of spouse and moving house changes one’s social resources. Membership of the LAC had benefits especially at these times and these events were often motivators to join such a club. Most participants found that their social resources declined after retirement and even felt that they were grieving for the loss of their work.

“ I just saw work as a collection of, um, colleagues as opposed to friends. I had a few good friends there. Most were simply colleagues or acquaintances …. [interviewer- Mmm.] ..Okay, you’d talk to them every day. You’d chatter in the kitchen, oh, pass banter back and forth when things are busy or quiet, but... Um, in terms of a friendship with those people, like going to their home, getting to know them, doing other things with them, very few. But what I did miss was the interaction with other people. It had simply gone….. But, yeah, look, that, the, yeah, that intervening period was, oh, a couple of months. That was a bit tough…. But in that time the people in LAC and the people in U3A…. And the other dance group just drew me into more things. Got to know more people. So once again, yeah, reasonable group of acquaintances.” (Male, PAFG)

Group members indicated general agreement with these two responses, however one female found she had a greater social life following retirement due to the busy nature of her job.

Within the social resources theme, three subthemes were identified, i) Opportunity for social connectedness, ii) Opportunity for friendships, and iii) Opportunity for social responsibility/leadership . Interestingly, these subthemes were additional to the information gathered in the survey. This emphasises the power of the inductive nature of the qualitative exploration employed in the focus groups to broaden the knowledge in this area.

The most discussed and expanded subtheme in both focus groups was Opportunity for social connectedness , which arose through developing new connections, diversifying social connections, sharing interests and experiences with others and peer learning. Participants in both focus groups stated that being a member of LAC facilitated their socialising and connecting with others to share ideas, skills and to do activities with, which was especially important through times of significant life events. Furthermore, participants in each of the focus groups valued developing diverse connections:

“ Yeah, I think, as I said, I finished up work and I, and I had more time for wa-, walking. So I think a, in meeting, in going to this group which, I saw this group of women but then someone introduced me to them. They were just meeting, just meeting a new different set of people, you know? As I said, my work people and these were just a whole different group of women, mainly women. There’s not many men. [Interviewer: Yes.]….. Although our leader is a man, which is ironic and is about, this man out in front and there’s about 20 women behind him, but, um, so yeah, and people from different walks of life and different nationalities there which I never knew in my work life, so yeah. That’s been great. So from that goes on other things, you know, you might, uh, other activities and, yeah, people for coffee and go to the pictures or something, yeah. That’s great.” (Female, PAFG)

Simply making new connections was the most widely discussed aspect related to the opportunity for social connectedness subtheme, with all participants agreeing that this was an important benefit of participation in LAC groups.

“Well, my experience is very similar to everybody else’s…….: I, I went from having no social life to a social life once I joined a group.” (Female, PAFG)

There was agreement in both focus groups that these initial new connections made at a LAC are strengthened through development of deeper personal connections with others who have similar demographics and who are interested in the same activities. This concurs with the Social Identity Theory [ 58 ] discussed previously.

“and I was walking around the lake in Ballarat, like wandering on my own. I thought, This is ridiculous. I mean, you’ve met all those groups of women coming the opposite way, so I found out what it was all about, so I joined, yeah. So that’s how I got into that.[ Interviewer: Yeah.] Basically sick of walking round the lake on my own. [Interviewer: Yeah, yeah.] So that’s great. It’s very social and they have coffee afterwards which is good.” (female, PAFG)

The subtheme Opportunity for development of friendships describes how, for some people, a number of LAC members have progressed from being just initial social connections to an established friendship. This signifies the strength of the connections that may potentially develop through LAC membership. Some participants from each group mentioned friendships developing, with slightly more discussion of this seen in the social group.

“we all have a good old chat, you know, and, and it’s all about friendship as well.” (female, SocialFG)

The subtheme Opportunity for social responsibility or leadership was mentioned by two people in the active group, however it was not brought up in the social group. This opportunity for leadership is linked with the development of a group identity and desiring to contribute meaningfully to a valued group.

“with our riding group, um, you, a leader for probably two rides a year so you’ve gotta prepare for it, so some of them do reccie rides themselves, so, um, and also every, uh, so that’s something that’s, uh, a responsibility.” (male, PAFG)

Social wellbeing

The social resources described above seem to contribute to a number of social, wellbeing outcomes for participants. The sub themes identified for Social wellbeing were , i) Increased social support, ii) Reduced loneliness, iii) Improved home relationships and iv) Improved social skills.

Increased social support

Social support was measured quantitatively in the survey (no significant change over time for new members) and identified as a benefit of LAC membership during the focus group discussions. However, only one of the members of the active group mentioned social support directly.

‘it’s nice to be able to pick up the phone and share your problem with somebody else, and that’s come about through LAC. ……‘Cos before that it was through, with my family (female, PAFG)

There was some agreement amongst participants of the PA group that they felt this kind of support may develop in time but most of them had been members for less than 2 years.

“[Interviewer: Yeah. Does anyone else have that experience? (relating to above quote)]” There is one lady but she’s actually the one that I joined with anyway. [Interviewer: Okay.] But I, I feel there are others that are definitely getting towards that stage. It’s still going quite early days. (female1, PAFG) [Interviewer: I guess it’s quite early for some of you, yeah.] “yeah” (female 2, PAFG)

Social support through sharing of skills was mentioned by one participant in the social group also, with agreement indicated by most of the others in the social focus group.

Discussion in the focus groups also touched on the subthemes Reduced loneliness and Improved home relationships, which were each mentioned by one person. And focus groups also felt that group membership Improved social skills through opening up and becoming more approachable (male, PAFG) or enabling them to become more accepting of others’ who are different (general agreement in Social FG).

This case study integrated results from a one-year longitudinal survey study and focus group discussions to gather rich information regarding the potential changes in social wellbeing that older adults may experience when joining community organisations offering group activities. The findings from this study indicate that becoming a member of such a community organisation can be associated with a range of social benefits for older adults, particularly related to reducing loneliness and maintaining social connections.

Joining a LAC was associated with a reduction in loneliness over 1 year. This finding is in line with past group-intervention studies where social activity groups were found to assist in reducing loneliness and social isolation [ 49 ]. This systematic review highlighted that the majority of the literature explored the effectiveness of group activity interventions for reducing severe loneliness or loneliness in clinical populations [ 49 ]. The present study extends this research to the general older adult population who are not specifically lonely and reported to be of good general health, rather than a clinical focus. Our findings are in contrast to results from an evaluation of a community capacity-building program aimed at reducing social isolation in older adults in rural Australia [ 59 ]. That program did not successfully reduce loneliness or improve social support. The lack of change from pre- to post-program in that study was reasoned to be due to sampling error, unstandardised data collection, and changes in sample characteristics across the programs [ 59 ]. Qualitative assessment of the same program [ 59 ] did however suggest that participants felt it was successful in reducing social isolation, which does support our findings.

Changes in loneliness were not a main discussion point of the qualitative component of the current study, however some participants did express that they felt less lonely since joining LACVI and all felt they had become more connected with others. This is not so much of a contrast in results as a potential situational issue. The lack of discussion of loneliness may have been linked to the common social stigma around experiencing loneliness outside certain accepted circumstances (e.g. widowhood), which may lead to underreporting in front of others [ 45 ].

Overall, both components of the study suggest that becoming a member of an activity group may be associated with reductions in loneliness, or at least a greater sense of social connectedness. In addition to the social nature of the groups and increased opportunity for social connections, another possible link between group activity and reduced loneliness is an increased opportunity for time out of home. Previous research has found that more time away from home in an average day is associated with lower loneliness in older adults [ 60 ]. Given the significant health and social problems that are related to loneliness and social isolation [ 13 , 14 , 15 ], the importance of group involvement for newly retired adults to prevent loneliness should be advocated.

In line with a significant reduction in loneliness, there was also a trend ( p  = 0.056) toward an increase in social support from baseline to 12 months in the survey study. Whilst suggestive of a change, it is far less conclusive than the findings for loneliness. There are a number of possible explanations for the lack of statistically significant change in this variable over the course of the study. The first is the small sample size, which would reduce the statistical power of the study. It may be that larger studies are required to observe changes in social support, which are possibly only subtle over the course of 1 year. This idea is supported by a year-long randomised controlled trial with 90 mildly-depressed older adults who attended senior citizen’s club in Norway [ 37 ]. The study failed to see any change in general social support in the intervention group compared to the control over 1 year. Additional analysis in that study suggested that people who attended the intervention groups more often, tended to have greater increases in SS ( p  = 0.08). The researchers stated that the study suffered from significant drop-out rates and low power as a result. In this way, it was similar to our findings and suggests that social support studies require larger numbers than we were able to gain in this early exploratory study. Another possible reason for small changes in SS in the current study may be the type of SS measured. The scale used gathered information around functional support or support given to individuals in times of need. Maybe it is not this type of support that changes in such groups but more specific support such as task-specific support. It has been observed in other studies and reviews that task-specific support changes as a result of behavioural interventions (e.g. PA interventions) but general support does not seem to change in the time frames often studied [ 61 , 62 , 63 ].

There were many social wellbeing benefits such as increased social connectivity identified in focus group discussion, but the specific theme of social support was rarely mentioned. It may be that general social support through such community groups may take longer than 1 year to develop. There is evidence that strong group ties are sequentially positively associated between social identification and social support [ 34 ], suggesting that the connections formed through the groups may lead increased to social support from group members in the future. This is supported by results from the focus group discussions, where one new member felt she could call on colleagues she met in her new group. Other new members thought it was too soon for this support to be available, but they could see the bonds developing.

Other social wellbeing changes

In addition to social support and loneliness that were the focus of the quantitative study, the focus group discussions uncovered a number of other benefits of group membership that were related to social wellbeing (see Fig. 3 ). The social resources theme was of particular interest because it reflected some of the mechanisms that appeared enable social wellbeing changes as a result of being a member of a LAC but were not measured in the survey. The main social resources relating to group membership that were mentioned in the focus groups were social connectedness, development of friendships and opportunity for social responsibility or leadership. As mentioned above, there was wide-spread discussion within the focus groups of the development of social connections through the clubs. Social connectedness is defined as “the sense of belonging and subjective psychological bond that people feel in relation to individuals and groups of others.” ([ 25 ], pp1). As well as being an important predecessor of social support, greater social connectedness has been found to be highly important for the health of older adults, especially cognitive and mental health [ 26 , 32 , 34 , 35 , 64 ]. One suggested theory for this health benefit is that connections developed through groups that we strongly identify with are likely to be important for the development of social identity [ 34 ], defined by Taifel as: “knowledge that [we] belong to certain social groups together with some emotional and value significance to [us] of this group membership” (Tajfel, 1972, p. 31 in [ 58 ] p 2). These types of groups to which we identify may be a source of “personal security, social companionship, emotional bonding, intellectual stimulation, and collaborative learning and……allow us to achieve goals.” ([ 58 ] p2) and an overall sense of self-worth and wellbeing. There was a great deal of discussion relating to the opportunity for social connectedness derived through group membership being particularly pertinent following a significant life event such as moving to a new house or partners becoming unwell or dying and especially retirement. This change in their social circumstance is likely to have triggered the need to renew their social identity by joining a community group. Research with university students has shown that new group identification can assist in transition for university students who have lost their old groups of friends because of starting university [ 65 ]. In an example relevant to older adults, maintenance or increase in number of group memberships at the time of retirement reduced mortality risk 8 years later compared to people who reduce their number of group activities in a longitudinal cohort study [ 66 ]. This would fit with the original Activity Theory of ageing; whereby better ageing experience is achieved when levels of social participation are maintained, and role replacement occurs when old roles (such as working roles) must be relinquished [ 67 ]. These connections therefore appear to assist in maintaining resilience in older adults defined as “the ability to maintain or improve a level of functional ability (a combination of intrinsic physical and mental capacity and environment) in the face of adversity” (p29, [ 5 ]). Factors that were mentioned in the focus groups as assisting participants in forming connections with others were shared interest, learning from others, and a fun and accepting environment. It was not possible to assess all life events in the survey study. However, since the discussion from the focus groups suggested this to be an important motivator for joining clubs and potentially a beneficial time for joining them, it would be worth exploring in future studies.

Focus group discussion suggested that an especially valuable time for joining such clubs was around retirement, to assist with maintaining social connectivity. The social groups seem to provide social activity and new roles for these older adults at times of change. It is not necessarily important for all older adults but maybe these ones identify themselves as social beings and therefore this maintenance of social connection helps to continue their social role. Given the suggested importance of social connectivity gained through this organisation, especially at times of significant life events, it would valuable to investigate this further in future and consider encouragement of such through government policy and funding. The majority of these types of clubs exist for older adults in general, but this study emphasises the need for groups such as these to target newly retired individuals specifically and to ensure that they are not seen as ‘only for old people’.

Strengths and limitations

The use of mixed –methodologies, combining longitudinal survey study analysed quantitatively, with a qualitative exploration through focus group discussions and thematic analysis, was a strength of the current study. It allowed the researchers to not only examine the association between becoming a member of a community group on social support and loneliness over an extended period, but also obtain a deeper understanding of the underlying reasons behind any associations. Given the variability of social support definitions in research [ 17 ] and the broad area of social wellbeing, it allowed for open exploration of the topic, to understand associations that may exist but would have otherwise been missed. Embedding the research in an existing community organisation was a strength, although with this also came some difficulties with recruitment. Voluntary coordination of the community groups meant that informing new members about the study was not always feasible or a priority for the volunteers. In addition, calling for new members was innately challenging because they were not yet committed to the club fully. This meant that so some people did not want to commit to a year-long study if they were not sure how long they would be a member of the club. This resulted in slow recruitment and a resulting relatively low sample size and decreased power to show significant statistical differences, which is a limitation of the present study. However, the use of Linear Mixed Models for analysis of the survey data was a strength because it was able to include all data in the analyses and not remove participants if one time point of data was missing, as repeated measures ANOVAs would do. The length of the study (1 year) is another strength, especially compared to previous randomised controlled studies that are typically only 6–16 weeks in length. Drop-out rate in the current study is very low and probably attributable to the benefits of working with long-standing organisations.

The purpose of this study was to explore in detail whether there are any relationships between joining existing community groups for older adults and social wellbeing. The lack of existing evidence in the field meant that a small feasibility-type case study was a good sounding-board for future larger scale research on the topic, despite not being able to answer questions of causality. Owing to the particularistic nature of case studies, it can also be difficult to generalise to other types of organisations or groups unless there is a great deal of similarity between them [ 68 ]. There are however, other types of community organisations in existence that have a similar structure to LACVI (Seniors centres [ 36 , 40 ], Men’s Sheds [ 38 ], University of the Third Age [ 34 , 69 ], Japanese salons [ 70 , 71 ]) and it may be that the results from this study are transferable to these also. This study adds to the literature around the benefits of joining community organisations that offer social and physical activities for older adults and suggests that this engagement may assist with reducing loneliness and maintaining social connection, especially around the time of retirement.

Directions for future research

Given that social support trended toward a significant increase, it would be useful to repeat the study on a larger scale in future to confirm this. Either a case study on a similar but larger community group or combining a number of community organisations would enable recruitment of more participants. Such an approach would also assist in assessing the generalisability of our findings to other community groups. Given that discussions around social benefits of group membership in the focus groups was often raised in conjunction with the occurrence of significant life events, it would be beneficial to include a significant life event scale in any future studies in this area. The qualitative results also suggest that it would be useful to investigate whether people who join community groups in early years post retirement gain the same social benefits as those in later stages of retirement. Studies investigating additional health benefits of these community groups such as physical activity, depression and general wellbeing would also be warranted.

With an ageing population, it is important to investigate ways to enable older adults to age successfully to ensure optimal quality of life and minimisation of health care costs. Social determinants of health such as social support, loneliness and social contact are important contributors to successful ageing through improvements in cognitive health, quality of life, reduction in depression and reduction in mortality. Unfortunately, older adults are at risk of these social factors declining in older age and there is little research investigating how best to tackle this. Community groups offering a range of activities may assist by improving social connectedness and social support and reducing loneliness for older adults. Some factors that may assist with this are activities that encourage sharing interests, learning from others, and are conducted in a fun and accepting environment. Such groups may be particularly important in developing social contacts for newly retired individuals or around other significant life events such as moving or illness of loved ones. In conclusion, ageing policy and strategies should emphasise participation in community groups especially for those recently retired, as they may assist in reducing loneliness and increasing social connections for older adults.

Abbreviations

Focus group

Life Activities Club

Life Activities Clubs Victoria

Linear mixed model

Physical activity

World Health Organisation

United Nations. In: P.D. Department of Economic and Social Affairs, editor. World Population Ageing 2015. New York: United Nations; 2015.

Google Scholar  

World Health Organisation. Global Health and Ageing. 2011 [cited 2014 March 25]; Available from: http://www.who.int/ageing/publications/global_health/en/ .

Balogun JA, et al. Age-related changes in balance performance. Disabil Rehabil. 1994;16(2):58–62.

Article   CAS   Google Scholar  

Singh MAF. Exercise comes of age: rationale and recommendations for a geriatric exercise prescription. J Gerontol Ser A Biol Med Sci. 2002;57(5):M262–82.

Article   Google Scholar  

World Health Organisation. World report on ageing and health. Geneva: World Health Organisation; 2015.

Rowe JW, Kahn RL. Successful Aging1. The Gerontologist. 1997;37(4):433–40.

Depp CA, Jeste DV. Definitions and predictors of successful aging: a comprehensive review of larger quantitative studies. Am J Geriatr Psychiatry. 2006;14(1):6–20.

Song M, Kong E-H. Older adults’ definitions of health: a metasynthesis. Int J Nurs Stud. 2015;52(6):1097–106.

Tate RB, Lah L, Cuddy TE. Definition of successful aging by elderly Canadian males: the Manitoba follow-up study. The Gerontologist. 2003;43(5):735–44.

Phelan EA, et al. Older Adults’ views of “successful aging”—how do they compare with Researchers’ definitions? J Am Geriatr Soc. 2004;52(2):211–6.

Haslam SA, et al. Social cure, what social cure? The propensity to underestimate the importance of social factors for health. Soc Sci Med. 2018;198:14–21.

Holt-Lunstad J, Smith TB, Layton JB. Social relationships and mortality risk: a meta-analytic review. PLoS Med. 2010;7(7):e1000316.

Richard A, et al. Loneliness is adversely associated with physical and mental health and lifestyle factors: results from a Swiss national survey. PLoS One. 2017;12(7):1–18.

Luo Y, et al. Loneliness, health, and mortality in old age: a national longitudinal study. Soc Sci Med. 2012;74(6):907–14.

Luo Y, Waite LJ. Loneliness and mortality among older adults in China. J Gerontol B Psychol Sci Soc Sci. 2014;69(4):633–45.

de Jong Gierveld J, Van Tilburg T, Dykstra PA. Loneliness and social isolation. In: Vangelisti A, Perlman D, editors. Cambridge handbook of personal relationships. Cambridge: Cambridge University Press; 2006. p. 485-500.

Williams P, Barclay L, Schmied V. Defining social support in context: a necessary step in improving research, intervention, and practice. Qual Health Res. 2004;14(7):942–60.

Valtorta N, Hanratty B. Loneliness, isolation and the health of older adults: do we need a new research agenda? J R Soc Med. 2012;105(12):518–22.

Jylhä M. Old age and loneliness: cross-sectional and longitudinal analyses in the Tampere longitudinal study on aging. Can J Aging La Revue can du vieil. 2010;23(2):157–68.

Huxhold O, Miche M, Schüz B. Benefits of having friends in older ages: differential effects of informal social activities on well-being in middle-aged and older adults. J Gerontol B Psychol Sci Soc Sci. 2014;69(3):366–75.

Berkman L, Syme S. Social networks, host resistance and mortality: a nine year follow-up study of alameda county residents. Am J Epidemiol. 1979;185(11):1070–88.

House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540–5.

Pynnonen K, et al. Does social activity decrease risk for institutionalization and mortality in older people? J Gerontol B Psychol Sci Soc Sci. 2012;67(6):765–74.

Broadhead WE, et al. The epidemiologic evidence for a relationship between social support and health. Am J Epidemiol. 1983;117(5):521–37.

Haslam C, et al. Social connectedness and health. Encyclopedia of geropsychology. 2017:2174–82. https://doi.org/10.1007/978-981-287-080-3_46-1 .

Haslam C, Cruwys T, Haslam SA. “The we’s have it”: evidence for the distinctive benefits of group engagement in enhancing cognitive health in aging. Soc Sci Med. 2014;120:57–66.

Uebelacker LA, et al. Social support and physical activity as moderators of life stress in predicting baseline depression and change in depression over time in the Women’s Health Initiative. Soc Psychiatry Psychiatr Epidemiol. 2013;48(12):1971–82.

Tajvar M, et al. Social support and health of older people in middle eastern countries: a systematic review. Australas J Ageing. 2013;32(2):71–8.

Dalgard OS, Bjork S, Tambs K. Social support, negative life events and mental health. Br J Psychiatry. 1995;166(1):29–34.

Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull. 1985;98(2):310–57.

Gilmour H. Social participation and the health and well-being of Canadian seniors. Health Rep. 2012;23(4):1B.

Glei DA, et al. Participating in social activities helps preserve cognitive function: an analysis of a longitudinal, population-based study of the elderly. Int J Epidemiol. 2005;34(4):864–71.

Lee SH, Kim YB. Which type of social activities may reduce cognitive decline in the elderly?: a longitudinal population-based study. BMC Geriatr. 2016;16(1):165.

Haslam C, et al. Group ties protect cognitive health by promoting social identification and social support. J Aging Health. 2016;28(2):244–66.

Haslam C, et al. Groups 4 health: evidence that a social-identity intervention that builds and strengthens social group membership improves mental health. J Affect Disord. 2016;194:188–95.

Bøen H. Characteristics of senior Centre users -- and the impact of a group programme on social support and late-life depression. Norsk Epidemiologi. 2012;22(2):261–9.

Bøen H, et al. A randomized controlled trial of a senior Centre group programme for increasing social support and preventing depression in elderly people living at home in Norway. BMC Geriatr. 2012;12:20.

Golding BG. Social, local, and situated: recent findings about the effectiveness of older Men’s informal learning in community contexts. Adult Educ Q. 2011;61(2):103.

Life Activities Clubs. Life Activities Clubs. About Us. . 2014 [cited 2014 January 13, 2014]; Available from: http://www.life.org.au/aboutus .

Hutchinson SL, Gallant KA. Can senior Centres be contexts for aging in third places? J Leis Res. 2016;48(1):50–68.

Millard J. The health of older adults in community activities. Work Older People. 2017;21(2):90–9.

Creswell JW, Plano-Clark VL. Designing and conducting mixed methods research. Thousand Oaks, Calif: SAGE Publications; 2007. p. c2007.

Loeb S, Penrod J, Hupcey J. Focus groups and older adults: tactics for success. J Gerontol Nurs. 2006;32(3):32–8.

PubMed   Google Scholar  

Broadhead W, et al. The Duke-UNC functional social support questionnaire: measurement of social support in family medicine patients. Med Care. 1988;26(7):709–23.

De Jong Gierveld J, Van Tilburg T. Living arrangements of older adults in the Netherlands and Italy: Coresidence values and behaviour and their consequences for loneliness. J Cross Cult Gerontol. 1999;14(1):1–24.

de Jong-Gierveld J, Kamphuls F. The development of a Rasch-type loneliness scale. Appl Psychol Meas. 1985;9(3):289–99.

Tilburg Tv, Leeuw Ed. Stability of scale quality under various data collection procedures: a mode comparison on the ‘De Jong-Gierveld loneliness scale. Int J Public Opin Res. 1991;3(1):69–85.

Hughes ME, et al. A short scale for measuring loneliness in large surveys - results from two population-based studies. Res Aging. 2004;26(6):655–72.

Dickens AP, et al. Interventions targeting social isolation in older people: a systematic review. BMC Public Health. 2011;11:647.

Hawkley LC, et al. Loneliness predicts increased blood pressure: 5-year cross-lagged analyses in middle-aged and older adults. Psychol Aging. 2010;25(1):132.

Netz Y, et al. Loneliness is associated with an increased risk of sedentary life in older Israelis. Aging Ment Health. 2013;17(1):40–7.

Australian Bureau of Statistics. Australian and New Zealand Standard Classification of Occupations, 2013, Version 1.2. Canberra: Australian Bureau of Statistics; 2013.

Australian Bureau of Statistics, Australian Standard Geographical Classification (ASGC). Canberra: Australian Bureau of Statistics; 2011.

Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey - construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.

Sanson-Fisher RW, Perkins JJ. Adaptation and validation of the SF-36 health survey for use in Australia. J Clin Epidemiol. 51(11):961–7.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Patton MQ. Qualitative research & evaluation methods. 4 ed. Thousand Oaks, California: Sage Publications; 2015.

Haslam SA, et al. Social identity, health and well-being: an emerging agenda for applied psychology. Appl Psychol. 2009;58(1):1–23.

Bartlett H, et al. Preventing social isolation in later life: findings and insights from a pilot Queensland intervention study. Ageing Soc. 2013;33(07):1167–89.

Petersen J, et al. Time out-of-home and cognitive, physical, and emotional wellbeing of older adults: a longitudinal mixed effects model. PLoS One. 2015;10(10):e0139643.

Lindsay Smith G, et al. The association between social support and physical activity in older adults: a systematic review. Int J Behav Nutr Phys Act. 2017;14(1):56.

Sallis JF, et al. The development of scales to measure social support for diet and exercise behaviors. Prev Med. 1987;16(6):825–36.

Oka R, King A, Young DR. Sources of social support as predictors of exercise adherence in women and men ages 50 to 65 years. Womens Health Res Gender Behav Policy. 1995;1:161–75.

CAS   Google Scholar  

Greenaway KH, et al. From “we” to “me”: group identification enhances perceived personal control with consequences for health and well-being. J Pers Soc Psychol. 2015;109(1):53–74.

Iyer A, et al. The more (and the more compatible) the merrier: multiple group memberships and identity compatibility as predictors of adjustment after life transitions. Br J Soc Psychol. 2009;48(4):707–33.

Steffens NK, et al. Social group memberships in retirement are associated with reduced risk of premature death: evidence from a longitudinal cohort study. BMJ Open. 2016;6(2):e010164.

Havighurst RJ. Successful aging. Gerontol. 1961;1:8–13.

Yin R. Case study research: design and methods. Beverly Hills, CA: Sage publishing; 1994.

Merriam SB, Kee Y. Promoting community wellbeing: the case for lifelong learning for older adults. Adult Educ Q. 2014;64(2):128–44.

Hikichi H, et al. Social interaction and cognitive decline: results of a 7-year community intervention. Alzheimers Dement: Translat Res Clin Interv. 2017;3(1):23–32.

Hikichi H, et al. Effect of a community intervention programme promoting social interactions on functional disability prevention for older adults: propensity score matching and instrumental variable analyses, JAGES Taketoyo study. J Epidemiol Community Health. 2015;69(9):905–10.

Download references

The primary author contributing to this study (GLS) receives PhD scholarship funding from Victoria University. The other authors were funded through salaries at Victoria University.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due the ethics approval for this study not allowing open access to the individual participant data but are available from the corresponding author on reasonable request.

Author information

Authors and affiliations.

Institute for Health and Sport, Victoria University, Melbourne, Australia

Gabrielle Lindsay-Smith, Grant O’Sullivan, Rochelle Eime, Jack Harvey & Jannique G. Z. van Uffelen

School of Health and Life Sciences, Federation University, Ballarat, Australia

Rochelle Eime & Jack Harvey

Department of Movement Sciences, Physical Activity, Sports and Health Research Group, KU Leuven - University of Leuven, Leuven, Belgium

Jannique G. Z. van Uffelen

You can also search for this author in PubMed   Google Scholar

Contributions

GLS, RE and JVU made substantial contributions to the conception and design of the study. GLS and GOS supervised data collection for the surveys (GLS) and focus groups (GOS and GLS). GLS, GOS, RE, JH and JVU were involved in data analysis and interpretation. All authors were involved in drafting, the manuscript and approved the final version.

Corresponding author

Correspondence to Gabrielle Lindsay-Smith .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval to conduct this study was obtained from the Victoria University Human Research Ethics Committee (HRE14–071 [survey] and HRE15–291 [focus groups]). All survey participants provided informed consent to partake in the study prior to undertaking the first survey by clicking ‘agree’ in the survey software (online participants) or signing a written consent form (paper participants) after they had read all participant information relating to the study. Focus group participants also completed a written consent form after reading and understanding all participant information regarding the study.

Consent for publication

Focus group participants have given permission for their quotes to be used in de-identified form prior to undertaking the research.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Lindsay-Smith, G., O’Sullivan, G., Eime, R. et al. A mixed methods case study exploring the impact of membership of a multi-activity, multicentre community group on social wellbeing of older adults. BMC Geriatr 18 , 226 (2018). https://doi.org/10.1186/s12877-018-0913-1

Download citation

Received : 19 April 2018

Accepted : 10 September 2018

Published : 24 September 2018

DOI : https://doi.org/10.1186/s12877-018-0913-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social engagement
• Group activity

BMC Geriatrics

ISSN: 1471-2318

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Student Notebook

Mixed Methods Research

• Experimental Psychology
• Quantitative
• Statistical Analysis

Traditionally, there are three branches of methodology: quantitative (numeric data), qualitative (observational or interview data), and mixed methods (using both types of data). Psychology relies heavily on quantitative-based data analyses but could benefit from incorporating the advantages of both quantitative and qualitative methodologies into one cohesive framework. Mixed Methods (MM) ideally includes the benefits of both methods (Johnson, Onwuegbuzie, & Turner, 2007): Quantitative analyses employ descriptive and inferential statistics, whereas qualitative analyses produce expressive data that provide descriptive details (often in narrative form) to examine the study’s research objectives. Whereas quantitative data may be collected via measures such as self-reports and physiological tests, qualitative data are collected via focus groups, structured or semistructured interviews, and other forms (Creswell, 2013).

MM hypotheses differ in comparison with solely quantitative or qualitative research questions. Not only must the quantitative and qualitative data be integrated, but the hypotheses also must be integrated. MM practitioners promote the development of a theory-based set of three hypotheses. Hypotheses should be conducted a priori and be both logical and sequential research questions (for more information, see Onwuegbuzie & Leech, 2006). Specialists encourage researchers to construct three separate types of hypotheses for an MM research project. There can be more than three hypotheses but there must be at least one of each type. The first hypothesis should be quantitative and the second should be qualitative. The third hypothesis will be an MM hypothesis.

Integration of these data is often complex, even when there is a strong theoretical rationale for doing so. Data integration occurs when quantitative and qualitative are combined in a data set. There are multiple ways for this to occur, including triangulation, following a thread, and the mixed methods matrix (see O’Cathain, Murphy, & Nicholl, 2010, for a brief review). Yet understanding the overall reasoning for using MM and how to best combine the approaches in practice can help lessen the challenge of MM data integration (Bryman, 2006).

Types of MM Research

• There are dozens of MM designs, but for the purpose of this article, six MM designs will be presented:
• The sequential explanatory method employs two different data-collection time points; the quantitative data are collected first and the qualitative collected last.
• The sequential exploratory design is best for testing emergent theory because both types of data are interpreted during the data integration phase.
• The sequential transformative approach has no preference for sequencing of data collection and emphasizes theory.
• Concurrent triangulation is the ideal method for cross-validation studies and has only one point of data collection.
• The concurrent nested design is best used to gain perspectives on understudied phenomena.
• The concurrent transformative approach is theory driven and allows the researcher to examine phenomena on several different levels.

Strengths and Challenges of MM Research

An MM approach is helpful in that one is able to conduct in-depth research and, when using complementary MM, provide for a more meaningful interpretation of the data and phenomenon being examined (Teddlie & Tashakkori, 2003).  Another strength of MM is the dynamic between the qualitative and quantitative portions of the study. If the design is planned appropriately, each type of data can mirror the other’s findings, so the methodology can benefit many types of research. However, interpreting data using the MM framework can be complicated and time intensive given that the data and interpretations are often abstract. Additionally, conducting MM research requires training and mastery of the methodology, so there can be a learning curve for researchers who traditionally use only quantitative or qualitative methods. Sticking to the theory-based and evidence-based designs will aid in your understanding and interpretation of the data.

Qualitative Data Analysis

Qualitative coding is a multistep process that includes different types of analyses depending on the nature of your data. Codebooks are important before, during, and after qualitative coding due to the detailed nature of the qualitative data. It is also important to know your expected codes and themes in order to promote interrater reliability (Hruschka et al., 2004). Expected codes are based on the theoretical foundation of your project. I suggest including the expected codes and themes in your codebooks. As previously mentioned, research designs involving this type of data can vary greatly, but in general, the following is a framework of how to conduct a thematic data analysis: Know your data inside and out, generate codes, search for themes, and review themes with a research team (Braun & Clarke, 2006). For more detailed instructions on conducting a qualitative analysis, please refer to last month’s Student Notebook article (Heydarian, 2016).

Lessons Learned

From the start, the researcher or research team must have a clear idea of their resources and the pros and cons of each method. Researchers also must be flexible. I am interested in examining the factors that compose seeking health information online. To investigate this topic, I developed an online, two-part study. Information obtained from qualitative prompts was used to inform the development of a scale measuring health-information-seeking behavior online. The first study used MM, and the data collection occurred on Amazon Mechanical Turk, a marketplace where researchers can post their available studies. Potential participants are paid a small fee, and data collection usually is completed in less than a week. I expected to conduct magnitude coding — a type of qualitative coding that evaluates the emphasis of content — but instead I had to choose a more appropriate type of coding because the participants provided extremely brief responses.

In closing, the design of your study (quantitative, qualitative, or MM) should align with your training and your research objectives. MM has the potential to bring your research to the next level by combining the strengths of quantitative and qualitative methodologies.

Suggestions for Conducting MM Research

Be proficient in MM research by keeping up to date with the latest techniques, software, textbooks, and manuals.

Think “outside the box” and consider other data-analytic approaches that are not used in your field.

Choose the research design that best fits the hypotheses, and know the assumptions and limitations of that design.

Incorporate figures and tables into your qualitative codebook to deepen the conceptualizations for the coders and provide a few examples of already coded data in order to provide thorough instructions.

Create and use summary statements for each participant to help with the abstract portion of the analyses. Summary statements should be a few sentences that describe the participant’s statement and provide an overall gist of the available qualitative information.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 , 77–101. doi:10.1191/1478088706qp063oa

Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6 , 97–113. doi:10.1177/1468794106058877

Creswell, J. W. (2013). Research design: Qualitative, quantitative, and mixed methods approaches . Thousand Oaks, CA: Sage Publications.

Heydarian, N. (2016). Developing theory with the grounded-theory approach and thematic analysis. Observer, 29(4) , 38–39.

Hruschka, D. J., Schwartz, D., John, D. C. S., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods, 16 , 307–331. doi:10.1177/1525822X04266540

Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1 , 112–133. doi:10.1177/1558689806298224

O’Cathain, A., Murphy, E., & Nicholl, J. (2010). Three techniques for integrating data in mixed methods studies. BMJ, 341 , c4587. doi:10.1136/bmj.c4587

Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking research questions to mixed methods data analysis procedures 1. The Qualitative Report, 11 , 474–498.

Teddlie, C., & Tashakkori, A. (2003). Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 3–50). Thousand Oaks, CA: Sage Publications.

VERY RELEVANT AND COMPREHENSIVE TEXT ON MM ETHODS

The analysis of mixed methods is fairly comprehensive and educative especially for scholars and/researchers who are used to the traditional Qualitative and Quantitatve research as a stand alone methodologies. I feel like looking for a workshop sponsor so that I can share these ideas to our colleagues in African universities generally and Kenya in particular. Our postgraduate students have not yet embrased the use of mixed methods. Four of my own supervised doctoral students have successfully used th MMR.We should do much more!

I am currently pursuing my PhD and using mixed method. I am interested in this combination of research methods.

I have gained much from the source which clearly spells out the strengths of MM and its applicability in research.

Iam conducting a sequential explanatory mixed methods study in PhD Management and I have benefited a lot from combining quantitative and qualitative research approaches operating with what works best per given research probem.

APS regularly opens certain online articles for discussion on our website. Effective February 2021, you must be a logged-in APS member to post comments. By posting a comment, you agree to our Community Guidelines and the display of your profile information, including your name and affiliation. Any opinions, findings, conclusions, or recommendations present in article comments are those of the writers and do not necessarily reflect the views of APS or the article’s author. For more information, please see our Community Guidelines .

Please login with your APS account to comment.

About the Author

Allyson S. Hughes is a Health Psychology doctoral student at The University of Texas at El Paso. Her research examines judgment and decision-making concerning health decisions using Internet resources. She can be reached at [email protected] .

Research Briefs

Recent highlights from APS journals articles on learning about working memory, psychological measurement, patterns of implicit and explicit attitudes, and much more.

Careers Up Close: Jolynn Pek on Quantifying Uncertainty

Pek analyzes sources of uncertainty in statistical results and develops new approaches to practicing replicable psychological science.

Speaking Evidence to Policy

The Commission on Evidence-Based Policymaking has issued a report that points to significant new opportunities in research and leadership for psychological scientists with expertise in areas including judgment and decision making, analysis and research design, data privacy and ethics issues, and more.

Privacy Overview

• Privacy Policy

Home » Mixed Methods Research – Types & Analysis

Mixed Methods Research – Types & Analysis

Table of Contents

Mixed Methods Research

Mixed methods research is an approach to research that combines both quantitative and qualitative research methods in a single study or research project. It is a methodological approach that involves collecting and analyzing both numerical (quantitative) and narrative (qualitative) data to gain a more comprehensive understanding of a research problem.

Types of Mixed Research

There are different types of mixed methods research designs that researchers can use, depending on the research question, the available data, and the resources available. Here are some common types:

Convergent Parallel Design

This design involves collecting both qualitative and quantitative data simultaneously, analyzing them separately, and then merging the findings to draw conclusions. The qualitative and quantitative data are given equal weight, and the findings are integrated during the interpretation phase.

Sequential Explanatory Design

In this design, the researcher collects and analyzes quantitative data first, and then uses qualitative data to explain or elaborate on the quantitative findings. The researcher may use the qualitative data to clarify unexpected or contradictory results from the quantitative analysis.

Sequential Exploratory Design

This design involves collecting qualitative data first, analyzing it, and then collecting and analyzing quantitative data to confirm or refute the qualitative findings. Qualitative data are used to generate hypotheses that are tested using quantitative data.

Concurrent Triangulation Design

This design involves collecting both qualitative and quantitative data concurrently and then comparing the results to find areas of agreement and disagreement. The findings are integrated during the interpretation phase to provide a more comprehensive understanding of the research question.

Concurrent Nested Design

This design involves collecting one type of data as the primary method and then using the other type of data to elaborate or clarify the primary data. For example, a researcher may use quantitative data as the primary method and qualitative data as a secondary method to provide more context and detail.

Transformative Design

This design involves using mixed methods research to not only understand the research question but also to bring about social change or transformation. The research is conducted in collaboration with stakeholders and aims to generate knowledge that can be used to improve policies, programs, and practices.

Concurrent Embedded Design

Concurrent embedded design is a type of mixed methods research design in which one type of data is embedded within another type of data. This design involves collecting both quantitative and qualitative data simultaneously, with one type of data being the primary method and the other type of data being the secondary method. The secondary method is embedded within the primary method, meaning that it is used to provide additional information or to clarify the primary data.

Data Collection Methods

Here are some common data collection methods used in mixed methods research:

Surveys are a common quantitative data collection method used in mixed methods research. Surveys involve collecting standardized responses to a set of questions from a sample of participants. Surveys can be conducted online, in person, or over the phone.

Interviews are a qualitative data collection method that involves asking open-ended questions to gather in-depth information about a participant’s experiences, perspectives, and opinions. Interviews can be conducted in person, over the phone, or online.

Focus groups

Focus groups are a qualitative data collection method that involves bringing together a small group of participants to discuss a topic or research question. The group is facilitated by a researcher, and the discussion is recorded and analyzed for themes and patterns.

Observations

Observations are a qualitative data collection method that involves systematically watching and recording behavior in a natural setting. Observations can be structured or unstructured and can be used to gather information about behavior, interactions, and context.

Document Analysis

Document analysis is a qualitative data collection method that involves analyzing existing documents, such as reports, policy documents, or media articles. Document analysis can be used to gather information about trends, policy changes, or public attitudes.

Experimentation

Experimentation is a quantitative data collection method that involves manipulating one or more variables and measuring their effects on an outcome. Experiments can be conducted in a laboratory or in a natural setting.

Data Analysis Methods

Mixed methods research involves using both quantitative and qualitative data analysis methods to analyze data collected through different methods. Here are some common data analysis methods used in mixed methods research:

Quantitative Data Analysis

Quantitative data collected through surveys or experiments can be analyzed using statistical methods. Statistical analysis can be used to identify relationships between variables, test hypotheses, and make predictions. Common statistical methods used in quantitative data analysis include regression analysis, t-tests, ANOVA, and correlation analysis.

Qualitative Data Analysis

Qualitative data collected through interviews, focus groups, or observations can be analyzed using a variety of qualitative data analysis methods. These methods include content analysis, thematic analysis, narrative analysis, and grounded theory. Qualitative data analysis involves identifying themes and patterns in the data, interpreting the meaning of the data, and drawing conclusions based on the findings.

Integration of Data

The integration of quantitative and qualitative data involves combining the results from both types of data analysis to gain a more comprehensive understanding of the research question. Integration can involve either a concurrent or sequential approach. Concurrent integration involves analyzing quantitative and qualitative data at the same time, while sequential integration involves analyzing one type of data first and then using the results to inform the analysis of the other type of data.

Triangulation

Triangulation involves using multiple sources or types of data to validate or corroborate findings. This can involve using both quantitative and qualitative data or multiple qualitative methods. Triangulation can enhance the credibility and validity of the research findings.

Mixed Methods Meta-analysis

Mixed methods meta-analysis involves the systematic review and synthesis of findings from multiple studies that use mixed methods designs. This involves combining quantitative and qualitative data from multiple studies to gain a broader understanding of a research question.

How to conduct Mixed Methods Research

Here are some general steps for conducting mixed methods research:

• Identify the research problem: The first step is to clearly define the research problem and determine if mixed methods research is appropriate for addressing it.
• Design the study: The research design should include both qualitative and quantitative data collection and analysis methods. The specific design will depend on the research question and the purpose of the study.
• Collect data : Data collection involves collecting both qualitative and quantitative data through various methods such as surveys, interviews, observations, and document analysis.
• Analyze data: Both qualitative and quantitative data need to be analyzed separately and then integrated. Analysis methods may include coding, statistical analysis, and thematic analysis.
• Interpret results: The results of the analysis should be interpreted, taking into account both the quantitative and qualitative findings. This involves integrating the results and identifying any patterns, themes, or discrepancies.
• Draw conclusions : Based on the interpretation of the results, conclusions should be drawn that address the research question and objectives.
• Report findings: Finally, the findings should be reported in a clear and concise manner, using both quantitative and qualitative data to support the conclusions.

Applications of Mixed Methods Research

Mixed methods research can be applied to a wide range of research fields and topics, including:

• Education : Mixed methods research can be used to evaluate educational programs, assess the effectiveness of teaching methods, and investigate student learning experiences.
• Health and social sciences: Mixed methods research can be used to study health interventions, understand the experiences of patients and their families, and assess the effectiveness of social programs.
• Business and management: Mixed methods research can be used to investigate customer satisfaction, assess the impact of marketing campaigns, and analyze the effectiveness of management strategies.
• Psychology : Mixed methods research can be used to explore the experiences and perspectives of individuals with mental health issues, investigate the impact of psychological interventions, and assess the effectiveness of therapy.
• Sociology : Mixed methods research can be used to study social phenomena, investigate the experiences and perspectives of marginalized groups, and assess the impact of social policies.
• Environmental studies: Mixed methods research can be used to assess the impact of environmental policies, investigate public perceptions of environmental issues, and analyze the effectiveness of conservation strategies.

Examples of Mixed Methods Research

Here are some examples of Mixed-Methods research:

• Evaluating a school-based mental health program: A researcher might use a concurrent embedded design to evaluate a school-based mental health program. The researcher might collect quantitative data through surveys and qualitative data through interviews with students and teachers. The quantitative data might be analyzed using statistical methods, while the qualitative data might be analyzed using thematic analysis. The results of the two types of data analysis could be integrated to provide a comprehensive evaluation of the program’s effectiveness.
• Understanding patient experiences of chronic illness: A researcher might use a sequential explanatory design to investigate patient experiences of chronic illness. The researcher might collect quantitative data through surveys and then use the results of the survey to inform the selection of participants for qualitative interviews. The qualitative data might be analyzed using content analysis to identify common themes in the patients’ experiences.
• Assessing the impact of a new public transportation system : A researcher might use a concurrent triangulation design to assess the impact of a new public transportation system. The researcher might collect quantitative data through surveys and qualitative data through focus groups with community members. The results of the two types of data analysis could be triangulated to provide a more comprehensive understanding of the impact of the new transportation system on the community.
• Exploring teacher perceptions of technology integration in the classroom: A researcher might use a sequential exploratory design to investigate teacher perceptions of technology integration in the classroom. The researcher might collect qualitative data through in-depth interviews with teachers and then use the results of the interviews to develop a survey. The quantitative data might be analyzed using descriptive statistics to identify trends in teacher perceptions.

When to use Mixed Methods Research

Mixed methods research is typically used when a research question cannot be fully answered by using only quantitative or qualitative methods. Here are some common situations where mixed methods research is appropriate:

• When the research question requires a more comprehensive understanding than can be achieved by using only quantitative or qualitative methods.
• When the research question requires both an exploration of individuals’ experiences, perspectives, and attitudes, as well as the measurement of objective outcomes and variables.
• When the research question requires the examination of a phenomenon in its natural setting and context, which can be achieved by collecting rich qualitative data, as well as the generalization of findings to a larger population, which can be achieved through the use of quantitative methods.
• When the research question requires the integration of different types of data or perspectives, such as combining data collected from participants with data collected from stakeholders or experts.
• When the research question requires the validation of findings obtained through one method by using another method.
• When the research question involves studying a complex phenomenon that cannot be understood by using only one method, such as studying the impact of a policy on a community’s well-being.
• When the research question involves studying a topic that has not been well-researched, and using mixed methods can help provide a more comprehensive understanding of the topic.

Purpose of Mixed Methods Research

The purpose of mixed methods research is to provide a more comprehensive understanding of a research problem than can be obtained through either quantitative or qualitative methods alone.

Mixed methods research is particularly useful when the research problem is complex and requires a deep understanding of the context and subjective experiences of participants, as well as the ability to generalize findings to a larger population. By combining both qualitative and quantitative methods, researchers can obtain a more complete picture of the research problem and its underlying mechanisms, as well as test hypotheses and identify patterns that may not be apparent with only one method.

Overall, mixed methods research aims to provide a more holistic and nuanced understanding of the research problem, allowing researchers to draw more valid and reliable conclusions, make more informed decisions, and develop more effective interventions and policies.

Advantages of Mixed Methods Research

Mixed methods research offers several advantages over using only qualitative or quantitative research methods. Here are some of the main advantages of mixed methods research:

• Comprehensive understanding: Mixed methods research provides a more comprehensive understanding of the research problem by combining both qualitative and quantitative data, which allows for a more nuanced interpretation of the data.
• Triangulation : Mixed methods research allows for triangulation, which is the use of multiple sources of data to verify findings. This improves the validity and reliability of the research.
• Addressing limitations: Mixed methods research can address the limitations of qualitative or quantitative research by compensating for the weaknesses of each method.
• Flexibility : Mixed methods research is flexible, allowing researchers to adapt the research design and methods as needed to best address the research question.
• Validity : Mixed methods research can increase the validity of the research by using multiple methods to measure the same concept.
• Generalizability : Mixed methods research can improve the generalizability of the findings by using quantitative data to test the applicability of qualitative findings to a larger population.
• Practical applications: Mixed methods research is useful for developing practical applications, such as interventions or policies, as it provides a more comprehensive understanding of the research problem.

Limitations of Mixed Methods Research

Here are some of the main limitations of mixed methods research:

• Time-consuming: Mixed methods research can be time-consuming and may require more resources than using only one research method.
• Complex data analysis: Integrating qualitative and quantitative data can be challenging and requires specialized skills for data analysis.
• Sampling bias: Mixed methods research can be subject to sampling bias, particularly if the sampling strategies for the qualitative and quantitative components are not aligned.
• Validity and reliability: Mixed methods research requires careful attention to the validity and reliability of both the qualitative and quantitative data, as well as the integration of the two data types.
• Difficulty in balancing the two methods: Mixed methods research can be difficult to balance the qualitative and quantitative methods effectively, particularly if one method dominates the other.
• Theoretical and philosophical issues: Mixed methods research raises theoretical and philosophical questions about the compatibility of qualitative and quantitative research methods and the underlying assumptions about the nature of reality and knowledge.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Questionnaire – Definition, Types, and Examples

Case Study – Methods, Examples and Guide

Observational Research – Methods and Guide

Quantitative Research – Methods, Types and...

Qualitative Research Methods

Explanatory Research – Types, Methods, Guide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• Mixed Methods Research | Definition, Guide, & Examples

Mixed Methods Research | Definition, Guide, & Examples

Published on 4 April 2022 by Tegan George . Revised on 25 October 2022.

Mixed methods research combines elements of quantitative research and qualitative research in order to answer your research question . Mixed methods can help you gain a more complete picture than a standalone quantitative or qualitative study, as it integrates benefits of both methods.

Mixed methods research is often used in the behavioral, health, and social sciences, especially in multidisciplinary settings and complex situational or societal research.

• To what extent does the frequency of traffic accidents ( quantitative ) reflect cyclist perceptions of road safety ( qualitative ) in Amsterdam?
• How do student perceptions of their school environment ( qualitative ) relate to differences in test scores ( quantitative ) ?
• How do interviews about job satisfaction at Company X ( qualitative ) help explain year-over-year sales performance and other KPIs ( quantitative ) ?
• How can voter and non-voter beliefs about democracy ( qualitative ) help explain election turnout patterns ( quantitative ) in Town X?
• How do average hospital salary measurements over time (quantitative) help to explain nurse testimonials about job satisfaction (qualitative) ?

Table of contents

When to use mixed methods research, mixed methods research designs, benefits of mixed methods research, disadvantages of mixed methods research, frequently asked questions about mixed methods research.

Mixed methods research may be the right choice if your research process suggests that quantitative or qualitative data alone will not sufficiently answer your research question. There are several common reasons for using mixed methods research:

• Generalisability : Qualitative research usually has a smaller sample size , and thus is not generalisable . In mixed methods research, this comparative weakness is mitigated by the comparative strength of ‘large N’, externally valid quantitative research.
• Contextualisation: Mixing methods allows you to put findings in context and add richer detail to your conclusions. Using qualitative data to illustrate quantitative findings can help ‘put meat on the bones’ of your analysis.
• Credibility: Using different methods to collect data on the same subject can make your results more credible. If the qualitative and quantitative data converge, this strengthens the validity of your conclusions. This process is called triangulation .

As you formulate your research question , try to directly address how qualitative and quantitative methods will be combined in your study. If your research question can be sufficiently answered via standalone quantitative or qualitative analysis, a mixed methods approach may not be the right fit.

Keep in mind that mixed methods research doesn’t just mean collecting both types of data; you need to carefully consider the relationship between the two and how you’ll integrate them into coherent conclusions. Mixed methods can be very challenging to put into practice, so it’s a less common choice than standalone qualitative or qualitative research.

Prevent plagiarism, run a free check.

There are different types of mixed methods research designs . The differences between them relate to the aim of the research, the timing of the data collection , and the importance given to each data type.

As you design your mixed methods study, also keep in mind:

• Your research approach ( inductive vs deductive )
• Your research questions
• What kind of data is already available for you to use
• What kind of data you’re able to collect yourself.

Here are a few of the most common mixed methods designs.

Convergent parallel

In a convergent parallel design, you collect quantitative and qualitative data at the same time and analyse them separately. After both analyses are complete, compare your results to draw overall conclusions.

• On the qualitative side, you analyse cyclist complaints via the city’s database and on social media to find out which areas are perceived as dangerous and why.
• On the quantitative side, you analyse accident reports in the city’s database to find out how frequently accidents occur in different areas of the city.

In an embedded design, you collect and analyse both types of data at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.

This is a good approach to take if you have limited time or resources. You can use an embedded design to strengthen or supplement your conclusions from the primary type of research design.

Explanatory sequential

In an explanatory sequential design, your quantitative data collection and analysis occurs first, followed by qualitative data collection and analysis.

You should use this design if you think your qualitative data will explain and contextualise your quantitative findings.

Exploratory sequential

In an exploratory sequential design, qualitative data collection and analysis occurs first, followed by quantitative data collection and analysis.

You can use this design to first explore initial questions and develop hypotheses. Then you can use the quantitative data to test or confirm your qualitative findings.

‘Best of both worlds’ analysis

Combining the two types of data means you benefit from both the detailed, contextualised insights of qualitative data and the generalisable, externally valid insights of quantitative data. The strengths of one type of data often mitigate the weaknesses of the other.

For example, solely quantitative studies often struggle to incorporate the lived experiences of your participants, so adding qualitative data deepens and enriches your quantitative results.

Solely qualitative studies are often not very generalisable, only reflecting the experiences of your participants, so adding quantitative data can validate your qualitative findings.

Method flexibility

Mixed methods are less tied to disciplines and established research paradigms. They offer more flexibility in designing your research, allowing you to combine aspects of different types of studies to distill the most informative results.

Mixed methods research can also combine theory generation and hypothesis testing within a single study, which is unusual for standalone qualitative or quantitative studies.

Mixed methods research is very labour-intensive. Collecting, analysing, and synthesising two types of data into one research product takes a lot of time and effort, and often involves interdisciplinary teams of researchers rather than individuals. For this reason, mixed methods research has the potential to cost much more than standalone studies.

Differing or conflicting results

If your analysis yields conflicting results, it can be very challenging to know how to interpret them in a mixed methods study. If the quantitative and qualitative results do not agree or you are concerned you may have confounding variables , it can be unclear how to proceed.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organisations.

Triangulation in research means using multiple datasets, methods, theories and/or investigators to address a research question. It’s a research strategy that can help you enhance the validity and credibility of your findings.

Triangulation is mainly used in qualitative research , but it’s also commonly applied in quantitative research . Mixed methods research always uses triangulation.

These are four of the most common mixed methods designs :

• Convergent parallel: Quantitative and qualitative data are collected at the same time and analysed separately. After both analyses are complete, compare your results to draw overall conclusions. 
• Embedded: Quantitative and qualitative data are collected at the same time, but within a larger quantitative or qualitative design. One type of data is secondary to the other.
• Explanatory sequential: Quantitative data is collected and analysed first, followed by qualitative data. You can use this design if you think your qualitative data will explain and contextualise your quantitative findings.
• Exploratory sequential: Qualitative data is collected and analysed first, followed by quantitative data. You can use this design if you think the quantitative data will confirm or validate your qualitative findings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, October 25). Mixed Methods Research | Definition, Guide, & Examples. Scribbr. Retrieved 29 April 2024, from https://www.scribbr.co.uk/research-methods/mixed-methods/

Is this article helpful?

Tegan George

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• JRSM Short Rep
• v.4(6); 2013 Jun

Using mixed methods in health research

Shema tariq.

1 School of Health Sciences, City University London, EC1A 7QN, London, UK

Jenny Woodman

2 MRC Centre of Epidemiology for Child Health, UCL Institute of Child Health, WC1N 1EH, London, UK

Mixed methods research is the use of quantitative and qualitative methods in a single study or series of studies. It is an emergent methodology which is increasingly used by health researchers, especially within health services research. There is a growing literature on the theory, design and critical appraisal of mixed methods research. However, there are few papers that summarize this methodological approach for health practitioners who wish to conduct or critically engage with mixed methods studies. The objective of this paper is to provide an accessible introduction to mixed methods for clinicians and researchers unfamiliar with this approach. We present a synthesis of key methodological literature on mixed methods research, with examples from our own work and that of others, to illustrate the practical applications of this approach within health research. We summarize definitions of mixed methods research, the value of this approach, key aspects of study design and analysis, and discuss the potential challenges of combining quantitative and qualitative methods and data. One of the key challenges within mixed methods research is the successful integration of quantitative and qualitative data during analysis and interpretation. However, the integration of different types of data can generate insights into a research question, resulting in enriched understanding of complex health research problems.

Introduction

Mixed methods research is the use of quantitative and qualitative methods in one study. Research is often dichotomized as quantitative or qualitative. Quantitative research, such as clinical trials or observational studies, generates numerical data. On the other hand qualitative approaches tend to generate non-numerical data, using methods such as semi-structured interviews, focus group discussions and participant observation. Historically, quantitative methods have dominated health research. However, qualitative methods have been increasingly accepted by the health research community in the past two decades, with a rise in publication of qualitative studies. 1 As the value of qualitative approaches has been recognized, there has been a growing interest in combining qualitative and quantitative methods. A recent review of health services research within England has shown an increase in the proportion of studies classified as mixed methods from 17% in the mid-1990s to 30% in the early 2000s. 2 In this paper, we present a synthesis of key literature on mixed methods research, with examples from our own work and that of others to illustrate the practical applications of this approach. This paper is aimed at health researchers and practitioners who are new to the field of mixed methods research and may only have experience of either quantitative or qualitative approaches and methodologies. We wish to provide these readers with an accessible introduction to the increasingly popular methodology of mixed methods research. We hope this will help readers to consider whether their research questions might best be answered by a mixed methods study design, and to engage critically with health research that uses this approach.

The authors each independently carried out a narrative literature review and met to discuss findings. Literature was identified via searches of PubMed, Google and Google Scholar, and hand-searches of the Journal of Mixed Methods Research, with relevant publications selected after discussion. An important consideration was that papers either had a methodological focus or contained a detailed description of their mixed methods design. For PubMed and Google searches, similar terms were used. For example, the PubMed strategy consisted of title and abstract searches for: ((mixed methods) OR ((mixed OR (qualitative AND quantitative)) AND methods)). We also drew upon recommendations from mixed methods conferences and seminars, and reference lists from key publications.

What is mixed methods research?

The most widely accepted definition of mixed methods research is research that ‘focuses on collecting, analysing, and mixing both quantitative and qualitative data in a single study or a series of studies’. 3 Central to the definition is the use of both quantitative and qualitative methods in one study (or a series of connected studies). Separate quantitative and qualitative studies addressing the same research question independently would not be considered ‘mixed methods’ as there would be no integration of approaches at the design, analysis or presentation stage. A recent innovation in mixed methods research is the mixed methods systematic review, which sets out to systematically appraise both quantitative and qualitative literature on a subject area and then synthesize the findings.

Why are mixed methods approaches used?

The underlying assumption of mixed methods research is that it can address some research questions more comprehensively than by using either quantitative or qualitative methods alone. 3 Questions that profit most from a mixed methods design tend to be broad and complex, with multiple facets that may each be best explored by quantitative or qualitative methods. See Boxes 1 and ​ and2 2 for examples from our own work.

Examples of authors’ mixed methods research – JW.

Examples of authors’ mixed methods research – ST.

Usually, quantitative research is associated with a positivist stance and a belief that reality that can be measured and observed objectively. Most commonly, it sets out to test an a priori hypothesis and is therefore conventionally described as ‘deductive’. Strengths of quantitative research include its procedures to minimize confounding and its potential to generate generalizable findings if based on samples that are both large enough and representative. It remains the dominant paradigm in health research. However, this deductive approach is less suited to generating hypotheses about how or why things are happening, or explaining complex social or cultural phenomena.

Qualitative research most often comes from an interpretive framework and is usually informed by the belief that there are multiple realities shaped by personal viewpoints, context and meaning. In-depth qualitative research aims to provide a rich description of views, beliefs and meaning. It also tends to acknowledge the role of researcher and context in shaping and producing the data. Qualitative approaches are described as ‘inductive’ as questions are often open-ended with the analysis allowing hypotheses to emerge from data. High-quality qualitative research can generate robust theory that is applicable to contexts outside of the study area in question, helping to guide practitioners and policy-makers. 8 However, for research that aims to directly impact on policy and practice, the findings of qualitative research can be limited by the small sample sizes that are necessary for in-depth exploratory work and the consequent lack of generalizabilty.

Mixed methods research therefore has the potential to harness the strengths and counterbalance the weaknesses of both approaches and can be especially powerful when addressing complex, multifaceted issues such as health services interventions 9 and living with chronic illness. 10

There are many reasons why researchers choose to combine quantitative and qualitative methods in a study. 11 , 12 We list some common reasons below, using a hypothetical research question about adolescents’ adherence to anticonvulsant medication to illustrate real world applications.

• Complementarity: Using data obtained by one method to illustrate results from another. An example of this would be a survey of adolescents with epilepsy demonstrating poor levels of adherence. Semi-structured interviews with a sub-group of those surveyed may allow us to explore barriers to adherence.
• Development: Using results from one method to develop or inform the use of the other method. A focus group conducted with a group of adolescents with epilepsy may identify mobile phone technology as a potentially important tool in adherence support. We could then develop a mobile phone ‘app’ that reminds patients to take their medication and conduct an intervention study to assess its impact on adherence levels.
• Initiation: Using results from different methods specifically to look for areas of incongruence in order to generate new insights. An illustration of this would be a study exploring the discrepancy between reported adherence in clinic consultations and actual medication adherence. A review of case notes may find adherence levels of over 90% in a clinic population; however, semi-structured interviews with peer researchers may reveal lower levels of adherence and barriers to open discussion with clinicians.
• Expansion: Setting out to examine different aspects of a research question, where each aspect warrants different methods. We may wish to conduct a study that explores adherence more broadly. A large-scale survey of adolescents with epilepsy would provide information on adherence levels and associations whilst interviews and focus groups may allow us to engage with individual experiences of chronic illness and medication in adolescence.
• Triangulation: Using data obtained by both methods to corroborate findings. For example, we could conduct a clinical study measuring drug levels in individuals and documenting self-reported adherence. Qualitative methods such as video diaries may confirm adherence levels.

To this list we would also add political commitment. That is to say, researchers may recognize, and wish to deploy, the strengths of quantitative research in producing generalizable results but may also be committed to representing the voice of participants in their work.

Whatever the reasons for mixing methods, it is important that authors present these explicitly as it allows us to assess if a mixed methods study design is appropriate for answering the research question. 3 , 13

How is mixed methods research conducted?

When embarking on a mixed methods research project it is important to consider:

• the methods that will be used;
• the priority of the methods;
• the sequence in which the methods are to be used.

A wide variety of methods exists by which to collect both quantitative and qualitative data. Both the research question and the data required will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team’s skills and experience and time constraints.

Priority of methods relates to the emphasis placed on each method in the study. For instance, the study may be predominantly a quantitative study with a small qualitative component, or vice versa. Alternatively, both quantitative and qualitative methods and data may have equal weighting. The emphasis given to each component of the study will be driven mainly by the research question, the skills of the research team and feasibility.

Finally, researchers must decide when each method is to be used in the study. For instance a team may choose to start with a quantitative phase followed by a qualitative phase, or vice versa. Some studies use both quantitative and qualitative methods concurrently. Again the choice of when to use each method is largely dependent on the research question.

The priority and sequence of mixing methods have been elaborated in a typology of mixed methods research models. See Table 1 for typology and specific examples.

Examples of studies using mixed methods.

How is data analysed in a mixed methods project?

The most important, and perhaps most difficult, aspect of mixed methods research is integrating the qualitative and quantitative data. One approach is to analyse the two data types separately and to then undertake a second stage of analysis where the data and findings from both studies are compared, contrasted and combined. 19 The quantitative and qualitative data are kept analytically distinct and are analysed using techniques usually associated with that type of data; for example, statistical techniques could be used to analyse survey data whilst thematic analysis may be used to analyse interview data. In this approach, the integrity of each data is preserved whilst also capitalizing on the potential for enhanced understanding from combining the two data and sets of findings.

Another approach to mixed methods data analysis is the integrative strategy. 20 Rather than keeping the datasets separate, one type of data may be transformed into another type. That is to say that qualitative data may be turned into quantitative data (‘quantitizing’) or quantitative data may be converted into qualitative data (‘qualitizing’). 21 The former is probably the most common method of this type of integrated analysis. Quantitative transformation is achieved by the numerical coding of qualitative data to create variables that may relate to themes or constructs, allowing statements such as ‘six of 10 participants spoke of the financial barriers to accessing health care’. These data can then be combined with the quantitative dataset and analysed together. Transforming quantitative data into qualitative data is less common. An example of this is the development of narrative psychological ‘types’ from numerical data obtained by questionnaires. 22

Potential challenges in conducting mixed methods research

Despite its considerable strengths as an approach, mixed methods research can present researchers with challenges. 23 , 24

Firstly, combining methodologies has sometimes been seen as problematic because of the view that quantitative and qualitative belong to separate and incompatible paradigms. In this context, paradigms are the set of practices and beliefs held by an academic community at a given point in time. 25 Researchers subscribing to this view argue that it is neither possible nor desirable to combine quantitative and qualitative methods in a study as they represent essentially different and conflicting ways of viewing the world and how we collect information about it. 8 Other researchers take a more pragmatic view, believing that concerns about the incommensurability of worldviews can be set aside if the combination of quantitative and qualitative methods addresses the research question effectively. This pragmatic view informs much applied mixed methods research in health services or policy. 8

Secondly, combining two methods in one study can be time consuming and requires experience and skills in both quantitative and qualitative methods. This can mean, in reality, that a mixed methods project requires a team rather than a lone researcher in order to conduct the study rigorously and within the specified time frame. However, it is important that a team comprising members from different disciplines work well together, rather than becoming compartmentalized. 26 We believe that a project leader with experience in both quantitative and qualitative methods can act as an important bridge in a mixed methods team.

Thirdly, achieving true integration of the different types of data can be difficult. We have suggested various analytic strategies above but this can be hard to achieve as it requires innovative thinking to move between different types of data and make meaningful links between them. It is therefore important to reflect on the results of a study and ask if your understanding has been enriched by the combination of different types of data. If this is not the case then integration may not have occurred sufficiently. 23

Finally, many researchers cite the difficulty in presenting the results of mixed methods study as a barrier to conducting this type of research. 23 Researchers may decide to present their quantitative and qualitative data separately for different audiences. This strategy may involve a decision to publish additional work focusing on the interpretations and conclusions which come from comparing and contrasting findings from the different data types. See Box 1 for an example of this type of publication strategy. Many journals in the medical sciences have a distinct methodological base and relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. However, as the number of mixed methods studies increases in the health research literature we would expect researchers to feel more confident in the presentation of this type of work.

Many of the areas we explore in health are complex and multifaceted. Mixed methods research (combining quantitative and qualitative methods in one study) is an innovative and increasingly popular way of addressing these complexities. Although mixed methods research presents some challenges, in much the same way as every methodology does, this approach provides the research team with a wider range of tools at their disposal in order to answer a question. We believe that the production and integration of different types of data and the combination of skill sets in a team can generate insights into a research question, resulting in enriched understanding.

DECLARATIONS

Competing interests.

None declared

This work was funded by the Medical Research Council (MRC) [grant number: G0701648 to ST], and the MRC with the Economic and Social Research Council (ESRC) [grant number: G0800112 to JW]

Ethical approval

No ethical approval was required for this work

Contributorship

This work was conceived by both ST and JW who each carried out an independent literature review and collaborated on the structure and content of this report. ST wrote the manuscript with revisions and editing done by JW

Acknowledgements

We thank Professors Jonathan Elford and Ruth Gilbert for their comments on draft manuscripts

This article was submitted by the authors and peer reviewed by Geoffrey Harding

• Open access
• Published: 26 April 2024

Overcoming barriers to equality, diversity, inclusivity, and sense of belonging in healthcare education: the Underrepresented Groups’ Experiences in Osteopathic Training (UrGEnT) mixed methods study

• Jerry Draper-Rodi   ORCID: orcid.org/0000-0002-1900-6141 1 , 2 ,
• Hilary Abbey 1 ,
• John Hammond 3 ,
• Oliver T. Thomson 1 ,
• Kevin Brownhill 1 ,
• Andrew MacMillan 1 , 4 ,
• Yinka Fabusuyi 1 &
• Steven Vogel 1  

BMC Medical Education volume  24 , Article number:  468 ( 2024 ) Cite this article

181 Accesses

1 Altmetric

Metrics details

Individuals from minority groups have historically faced social injustices. Those from underrepresented groups have been less likely to access both healthcare services and higher education. Little is known about the experiences of underrepresented students during their undergraduate studies in osteopathy in the UK. The aim of this project was to explore awareness of cultural diversity and beliefs about patients from underrepresented groups in current osteopathic educational environments and evaluate students’ preparedness to manage patients from diverse groups. The project also aimed to investigate the educational experiences of students from underrepresented backgrounds during their training and their opinions on changes that could support better levels of recruitment and achievement. The findings were discussed with stakeholders in interactive workshops with the aim to develop recommendations for action and change.

A transformative action research paradigm informed this mixed methods project. It included: 1/ a survey of students from all seven osteopathic educational providers in the UK using the Multidimensional Cultural Humility Scale (MCHS); 2/ a series of focus groups with students from underrepresented groups (women, students with disabilities, students from minority ethnic backgrounds, and students identifying as LGBTQIA+); and 3/ a workshop forum to discuss findings.

A total of 202 participants completed the MCHS and demographic questionnaire and seven focus groups were conducted. A model was developed to describe participants’ training experiences comprising two main themes: institutional contextual obstacles (with four sub-themes) and underrepresented students’ conceptual understanding of Equity, Diversity and Inclusion (EDI). Recommendations for change identified in the workshops were based on three topics: institutions, staff, and students.

Our findings confirm conclusions from other institutions that staff education is urgently needed to create and maintain equitable, inclusive environments in osteopathic educational institutions in the UK to support all students, particularly those from underrepresented groups. Institutional EDI processes and policies also need to be clarified or modified to ensure their usefulness, accessibility, and implementation.

Peer Review reports

Social injustices affecting people from minority groups have been highlighted in recent worldwide initiatives such as the ‘Black Lives Matter’ [ 1 ] and ‘Me Too’ [ 2 ] movements and investigations have identified institutional racism, sexism and homophobia in the police, other public services, and business organisations [ 3 , 4 , 5 , 6 ]. Limited demographic diversity and evidence of discrimination against minority groups have been reported in higher education in the United Kingdom (UK) [ 7 ] and in healthcare services including medicine, psychiatry, and physiotherapy [ 8 , 9 , 10 ]. Data from higher education institutions suggest there is an urgent need to improve recruitment, educational experiences, and attainment for students from minority groups [ 11 , 12 , 13 ].

The terms ‘minority’ or ‘under-represented’ are often used interchangeably to describe groups of people identified by specific demographic or cultural characteristics. In this paper, the term ‘under-represented’ is used to emphasise that experiences of inequity are typically created and maintained by social constructs such as ‘othering’: the process of identifying people as different from oneself or the mainstream culture, often associated with negative beliefs and expectations [ 14 ]. Social constructs can provide both unearned advantage (‘privilege’) and disadvantage (‘oppression’) [ 15 ]. Characteristics used to identify others can include skin colour, ethnicity, religion, gender identity, sexual identity, ability, size, socioeconomic status, history of trauma, addiction, and family environment [ 15 ].

People from under-represented groups (UrGs) have historically been less likely to access higher education [ 16 ], although the number of BAME, LGBTQIA + and disabled students is gradually increasing in England [ 17 , 18 ]. Enrolled students from these groups are reported to experience more negative experiences during training and more limited later career opportunities afterwards [ 19 , 20 ]. The General Medical Council (GMC) recently set new targets to improve access and outcomes for students from UrGs [ 21 ] as lack of diversity and limited cultural awareness among practitioners from different healthcare professions also impacts the quality and outcomes of healthcare for patients from UrGs [ 12 , 22 , 23 ]. The Council of Deans recently published a report on how to build an inclusive environment which highlights issues that affect students from minority ethnic groups in Allied Health Professions [ 24 ].

Patients from UrG experience substantial health disparities in the UK and across the globe due to structural and interpersonal discrimination [ 25 , 26 ]. Developing cultural humility in clinicians is seen as key to bridging the gap of interpersonal discrimination. Cultural competence was once considered as an adequate way to provide an inclusive environment. It is defined as “a set of congruent behaviours, attitudes and policies that come together in a system, agency or among professionals that enable that system, agency or professions to work effectively in cross-cultural situations” ([ 27 ] p. iv). The concept shifted to cultural humility, defined as “the ability to maintain an interpersonal stance that is other-oriented (or open to the other) in relation to aspects of cultural identity that are most important to the client” ([ 28 ] p. 354).

Osteopathy is a form of manual therapy which is now recognised as one of 14 Allied Healthcare Professions in England [ 29 ]. In the UK, there are currently seven osteopathic education providers (OEPs) and approximately 5,300 qualified osteopaths. Training is typically over four or five years in the form of Bachelor’s or Integrated Masters awards and practitioners then register with the statutory regulator, the General Osteopathic Council (GOsC), and are required to comply with professional standards of practice [ 30 ].

There is little known about discrimination, bullying and harassment in osteopathy education as highlighted in a recent systematic review [ 31 ]. Therefore, the current research project aimed to assess osteopathic students’ awareness of cultural diversity and beliefs about patients from UrGs and their preparedness in managing them; to explore the educational experiences of students with UrG backgrounds during training and their opinions on changes to support better levels of recruitment and achievement. Finally, the research was disseminated to stakeholders in workshops with the overall aim of developing recommendations for action and creating change.

To meet the multiple aims, a mixed methods approach was implemented and included the following stages; a survey of students attending all seven OEPs in the UK; focus groups with UrG students; and a workshop forum to explore the findings with diverse stakeholders. This design was based on a transformative action research paradigm with students participating as collaborators (Mertens 2007; 2010), informed by previous research into EDI, cultural competence and cultural humility in healthcare education, outlined below. The research complies with the Good Reporting of A Mixed Methods Study (GRAMMS) guidance [ 32 ] (see supplementary material 1 – GRAMMS reporting).

Figure  1 below details the mixed method stages with the quantitative data collection (top half of figure), qualitative data collection (bottom half), and mixed methods stages (middle). The stages are represented chronologically, starting on the left.

Study design

Methodology

This research project sits within a transformative paradigm that places central importance on studying the lives and experiences of marginalised groups and is appropriate for addressing inequality and injustice in society [ 33 ]. An explanatory sequential mixed methods design (survey followed by focus groups) was implemented to gain insight [ 34 ] and community members were involved in initial discussions about operationalising the research focus. Transformative research has power issues and inequalities at its core and a political agenda that aims to change the experiences of the participants and institutions involved [ 35 ]. The study was approved by the University College of Osteopathy Research Ethics Committee.

Community engagement

Two community engagement meetings with students from underrepresented groups were established prior to the project to ensure it was designed ‘with’ students rather than ‘to’, ‘about’ or ‘for’ them. Based on principles by [ 36 ], these community engagement meetings co-created the study design, modified the research questionnaire and recruitment approaches.

Quantitative stage

A survey of all students currently enrolled on an osteopathic course in the UK was chosen to explore the research objectives. All students enrolled at the seven OEPs in the UK (excluding postgraduate and CPD courses) were eligible to take part in the anonymous online survey on Qualtrics©. Invitations, study information and accessible links were disseminated via OEP contacts who sent it to their student body between 7th and 31st March 2022. Two reminders were sent.

Survey instrument

The Multidimensional Cultural Humility Scale (MCHS) was selected for this project as there is good evidence of convergent and discriminant validity and internal reliability [ 37 ]. The MCHS has five dimensions, contains 15 items with a 6-point Likert scale from ‘strongly disagree’ to ‘strongly agree’ where higher scores represent greater levels of cultural humility. The MCHS was used to understand to understand awareness of cultural sensitivity in the environment in which UrG students were learning. This project was not about clinical services. Modifications to the MCHS were necessary to contextualise it for osteopathy students, so a factor analysis was conducted to assess the validity of the adapted version. Following the community engagement meetings, a 7th category was added: ‘This has never crossed my mind’ to assess whether students were comfortable, confident or aware of particular issues (see supplementary material 2 for the adapted versions used in this study).

Questions related to demographics and personal characteristics (clinical or pre-clinical student, age, birth sex, gender, ethnicity, health and disability status, sexual orientation, and religion), and to their experience of education were included at the end of the MCHS and were analysed separately to the MCHS questionnaire.

Qualitative stage

Focus groups were selected for this phase and represented four UrG: ethnic minority, disability, LGBTQIA + or women. Whilst women are not numerically under-represented in UK osteopathic undergraduate training, socially they are more oppressed than men, including in manual therapy training [ 31 ]. The choice of these four groups was discussed and agreed as important priorities in the community engagement meetings. For sensitive topics, homogeneous groups foster a sense of belonging and facilitate disclosure [ 38 ]. Focus groups usually comprise 6 to 8 people who meet once for approximately 90–120 min, and the usual number of groups is around 4 but depends on the complexity of the topic and heterogeneity of the samples [ 39 ].

Students from any UK OEP who identified as belonging to at least one UrG (ethnic minority, disability, LGBTQIA + or women) were eligible to participate with students from the same and/or other OEPs. Each OEP was responsible for forwarding invitations to participate to their students. For convenience, focus groups were conducted online as students from different OEPs were geographically dispersed [ 40 ]. The research team members acting as focus group facilitators identified with one or more minority groups, representing diversity and were therefore part of the data, as is good practice in transformative paradigmatic research [ 41 ]. All facilitators had previously used focus group methods, participated in training, or were used to managing student group discussions. Teams© created automatic initial draft transcriptions to aid later transcription if participants talked simultaneously [ 39 ]. Final transcripts only included pseudonyms, as is common in qualitative research [ 42 ]. Focus groups sessions ran for approximately 90 min. Students who had participated in one of the four initial groups were invited to join one final mixed group to discuss the previous findings, and students who participated in at least one group were invited to take part in the workshop forum.

Dissemination forum and discussion workshops

An interactive face-to-face workshop-based forum was held on 06/04/2023 to disseminate the survey and focus group results, discuss their implications, and develop recommendations for action. Key stakeholders invited to attend free of charge included UK OEPs, the General Osteopathic Council, the Institute of Osteopathy, the Osteopathic Foundation, and other healthcare profession organisations, NHS representatives, and Health Education England. Approximately 70 people attended the event. Three interactive workshops focusing on specific aspects of EDI (students, staff and institutional governance), with different methods to promote open discussion, explored responses about ways to develop a more supportive educational environment and inclusive curriculum.

Mixed methods analysis

To assess whether the 5-factor model of the MCHS remained valid following changes made to the scale, a confirmatory factor analysis was carried out using using R (version 4.3.2) [ 43 ]and the R lavaan package (version 0.6–16) [ 44 ]. Missing MCHS data was imputed using multivariate imputation by chained equations [ 45 ]. MCHS data was checked for normality using QQ plots and the Henze-Zirkler test.

A sum of all MCHS items (reverse coded as appropriate) was calculated as an overall measure of cultural humility. Linear regression was carried out to determine which demographic factors influenced this total score. Additionally, a Welch Two Sample t-test [ 46 ] was carried out to determine if MCHS total score differed between clinical and pre-clinical students. Chi-squared tests, with p-values estimated by Monte-Carlo simulation, were used to test for associations between students’ report of having been treated differently one the one hand, and demographic factors on the other. Descriptive statistics were used to report survey results.

Focus group data analysis was conducted within a reflexive thematic analysis framework [ 47 ], which aligns with a transformative paradigm (Creswell 2014). Data was co-created by participants and facilitators, and themes were co-created with analysts through their thoughtful engagement with the data [ 47 ]. After conducting one focus group with each UrG ( n  = 4), early analysis was conducted. Another 4 focus groups with different students were conducted to analyse how these participants’ experiences resonated with the initial findings. The last focus groups ran with students from mixed UrGs to discuss the findings, conduct a meta-synthesis, and prioritise what actions students thought OEPs should prioritise.

Three interactive workshops were run to explore the resonance and implications of the quantitative and qualitative findings to date. Each workshop focused on either student, staff or institutional EDI issues, although there was inevitably some overlap, and each workshop ran three times to enable participants to contribute fully. Small groups of mixed stakeholders worked took part in varied activities to discuss the study’s findings and their ideas were recorded on post-it notes, flipcharts or noted by facilitators during plenary discussions. After the workshop, written comments were collated by the facilitators (YF, HA, SV) and categorised into themes by members of the research team (JDR, HA), using frequency analysis (where data was available) to identify strong and recurring recommendations for change.

The data from the quantitative and qualitative phases were analysed separately, but then were considered together both in the forum workshops and within the research team. When considering the quantitative and qualitative datasets together, the research team operated within the methodological spirit of pragmatism, whereby both data sets were integrated in such a way that a useful insight to the research provided useful insights to participants’ experiences and generate knowledge with social utility [ 48 ]. In practice, this meant that survey results were presented to focus group participants to stimulate reflection and discussion and explore how the results compared with their personal experiences. Finally, the workshops provided an additional method to explore, situate and integrate the synthesised qualitative and quantitative data sets to support development of the final thematic model.

Quantitative results

Two hundred and two participants filled in the survey, of which 117 (58%) were complete. The response rate was 20% (Table  1 ). Responses per OEP ranged between 6 and 68 (Table  2 – OEP Responses).

Seventy percent of the respondents provided demographic information ( n  = 142). Participants were mostly white ( n  = 95), female ( n  = 74), without a disability ( n  = 106), heterosexual ( n  = 89), and identifying with no religion ( n  = 69) (see Table  3 – respondent demographics).

Most participants identified to some extent with an UrG ( n  = 62, 53%). Of all the students who responded (53% self-identifying as UrG to some extent, and 47% who did not identify as UrG), 67.8% ( n  = 80) reported that they had not been treated differently because of their cultural background or identity. Those who had been treated differently ( n  = 19; 16%) stated that it happened at least a few times per year ( n  = 15, 79%) (supplementary material 3 , table a– underrepresented groups treatment). Of the 28 who reported having been treated differently, 18 reported whether they had complained: 15 had not complained (6 open-ended responses: not significant enough ( n  = 2), unlikely to lead to change ( n  = 2), fear of being identified ( n  = 1), happened once and felt that mistakes happen ( n  = 1)). Six of the 15 who did not complain did not know how or to whom to complain.

Associations between demographic characteristics and UrG self-identification found that ethnicity (merging all categories excluding White), Disability and Sexual Orientation (merging all categories excluding heterosexual) were significantly associated with identifying as belonging to an UrG group (Supplementary material 3 , Table b - UrG identification vs. demographic group).

No significant associations were found between demographic characteristics and reports of being treated differently (Supplementary material 3 , table c - treated differently vs. demographic group).

Of the 19 participants who reported having been treated differently because of their culture or identity, 79% ( n  = 15) did not report it to their OEP, 15.8% ( n  = 3) did, and 5.2% ( n  = 1) did not answer.

It was not possible to confirm or deny the adequacy of the 5-factor model proposed by Gonzales et al. [ 37 ] (Supplementary material 4 ), so our analysis was based on their 5-factor model (see Table  4 – MCHS results). Regarding the MCHS total score, no differences were found between clinical and preclinical students (Welch’s t = -0.194, df = 79.3, p  = 0.847). A weak correlation between MCHS total score and importance to individual was found (Spearman’s rho(114) = 0.27, p  = 0.003), and a weak relationship between self-rating of skills and MCHS total score (rho(114) = 0.26, p  = 0.005). There was no apparent relationship between MCHS total score and participants’ perception of support in the clinical environment for exploring patients’ backgrounds and experiences (rho(106) = 0.097, p  = 0.3). No scores on these three questions differed significantly between clinical and preclinical students.

Qualitative results

Seven groups were conducted, each were facilitated by two members of the research team (from AMM, HA, JDR, SV, YF). Data from the first six focus groups were organised into two themes which provide descriptive insights of participants’ reflections on the quantitative findings and how these results related to and resonated with their own experiences. The two primary themes were named institutional contextual obstacles (with 4 sub-themes) and UrG students’ conceptual understanding of EDI (with 3 sub-themes). The themes and sub-themes were modelled and presented to the final focus group to facilitate reflective discussions, see Fig.  2 .

Model based on focus groups’ themes and sub-themes

Theme 1: Institutional contextual obstacles

The first sub-theme, Faculty’s lack of awareness & knowledge , was a commonly reported barrier.

I think there’s a lot of talk of self-reflection, at least at the OEP, and it doesn’t to me feel like all of our teachers practise that. I’ve had more problems with staff understanding than student understanding”. (Talking about their disability) There was no awareness, you know, of that within the class or from the tutors, in those circumstances (managing an LGBTQ + patient), what do we do, what language do we use, (…) when it was raised the tutor was sort of like, actually, I don’t have an answer, I’m not sure.

Racist, sexist and ableist comments made by staff negatively affected the way students interacted with patients in the OEPs clinics, and with other students, particularly in practical classes.

I was doing a neck and then teacher wants me to talk when I’m doing it and I say, because / when I’m doing it, I can’t talk and he made a comment, as a woman, you should talk and you should do it, you should multi-task and at that time I couldn’t say anything because I [was] already panicking and I’m doing this thing. I couldn’t say anything. [A male tutor] put [a female tutor acting as a model] side-lying and [he] was going to crack her back but then when he pulled her shirt up her scrubs pants were like mid-way / quite / kind of showing her underwear (…) When we told him that he should pull her scrubs up, he made the thing super uncomfortable. When they make an attack, as a joke, and people laugh, that’s positive behaviour, they’re going to make the joke again because it’s funny, so I don’t know if they can understand that it’s actually a knife that you’re throwing at someone and not just a joke.

The second sub-theme related to a lack of support from institutions for students from UrG, and a lack of clarity of processes available to them to complain about discriminatory behaviours against them.

When I was sort of going through the process of applying for the disabled students’ allowance, which I didn’t even know that I was / its existence to be honest, (…) I had to get the OEP to fill out a form and rubber stamp it and it seemed to get lost in this abyss of I don’t know where it went. (…) but there was a lot of chasing up to do [laughs] and even getting the form signed again, because I have to reapply every year, was a bit of a faff.

Participants who reported discrimination, were lacking certainty that reported these instances would lead to change.

Particularly when it’s a comment like that that’s made and it almost leaves you like gobsmacked and you’re like well what do I say to that, how do I go about telling someone about that?

The third sub-theme was Student attitudes e.g., peers making sexist comments and using negative language about UrGs.

People have said things, especially kind of bisexual tropes and things like that about you know being greedy and I know it’s / (…) people think oh that’s funny (…) it just makes you feel like you are going inward kind of thing. I was practicing thoracic HVT with (…) some first years [students] and I started doing thoracic HVT and one of the first years asked me to do it on him, so I was like, okay, umm, I explained to him you know everything, asked for his consent and stuff, but because he was like a funny guy, he was talking all the time, I was like, okay, can you just sit down for me to do the technique and I told him my nationality before that and then he goes like oh that’s how I know you are Brazilian, your attitude, you probably go on top. I’m just like what? You know / yes, I didn’t even know what to say at this time, because I was just / I just told him, look, I’m not doing the technique, I thought, goodbye.

Participants reported instances where students from privileged backgrounds remained silent when facing discriminatory comments from educational faculty; a factor that perpetuated a non-inclusive culture, as people who used discriminatory or ‘othering’ language were not challenged to reflect on their attitudes and behaviour. In contrast, participants from UrGs felt a sense of duty to raise concerns:

I don’t create problems and stuff, but if there is something if I see it not going right, I like to raise my voice as much as I can and I try to make changes.

The fourth sub-theme was Lack of representation in the student body, patient population and the curriculum.

Everything that we get taught is 99% on like a male sex anatomy. Like I remember when I was learning how to do all the like umm cardiac testing and respiratory we were taught by a male teacher on a male body and then when it came to like a female and like you have boobs and they’re like, oh, you can’t do this bit at the front, or you have to be more careful, but then there was no example of how. I think I felt surprised when coming into the / into osteopathy how less diverse (in student demographics? ) it is than my previous position. I feel quite diverse but people that we see in clinic are mainly Caucasian, so I also think there’s something about the outreach of osteopathy into different cultural communities, for example, most of my family, though we’ve all been brought up here, nobody would use an osteopath (…). When we learn about physiology and pathologies, I feel like there’s now a real effort to talk about say like black people, which is fantastic, but then you know what about Asian (…).

Theme 2: Underrepresented students’ understanding of EDI

The first sub-theme related to the definition of discrimination and echoed findings from community engagement discussions. Students distinguished between ‘othering’ and ‘intent’. Participants perceived discrimination only when actions had an intent to discriminate against individuals or communities, rather than actions that led to people or groups being treated differently regardless of intent. During the focus groups, participants reported equal treatment, but data analysis suggests instances of discrimination.

No, only in so much as, you know, the reasonable adjustments aspect, but then I’ll ask for that, but besides that, I haven’t / I haven’t had any different treatment. I’ve definitely been treated differently as a woman and / but I’ve witnessed the / in my class Asian women being treated differently, but the Asian men not so much so.

The second sub-theme related to the advocacy of UrG students as role models for their peers. Students used their own experience of belonging to an UrG as personal knowledge to help inform their peers about what it is like to be a person from wider UrG communities. This helped to fill gaps in the EDI training or make up for a lack of training received by educators. UrG students acted as advocates to prevent wrong messages, jokes being shared, e.g.,

I think it’s / not just from my disability, but yes, from / for all other students I think when they / things come up, sometimes quite surprising things actually, it’s usually / yes, pretty interesting and helpful for all of us. We use it [disability] sometimes in class as part of like chronic pain, as part of that kind of presentation and things like that because I have an understanding of it, whereas instead of just pulling stories out of thin air.

The third related sub-theme was that students from UrGs appeared to have a better understanding of EDI than their peers and faculty members. Students’ advocacy role included training and supporting their peers in how they should manage situations when facing patients with specific conditions, e.g. type 1 diabetes, and offered a useful insight which would be valued by patient.

I mean do they have to? Should they? I think you know, like I’ve said, the only reason I do [disclose] it is because you know I wouldn’t want to put anybody else in a tricky position if I was to, you know, have like a hypo in class or anything like that, which you know, I may do one day.

This created an environment where students from UrGs not only had to teach other students and faculty, but also had to learn on their own, as they were not able to gain knowledge from staff on topics related to UrG, and then had to teach what they learned to their peers and faculty members.

But we don’t get taught about how to deal with somebody that’s transgender or anything like that. It’s like well you’ll have to you know just find out about that yourself. I don’t have that much of an understanding of the difference that ethnicity has on sort of different diseases and different morphologies and things like that, so it’s something (…) I’d love to learn more about.

The final mixed focus group was used to explore whether the above findings represented the experiences of these participants, and to generate suggestions for OEP action to become more inclusive. Goals thought to be quickly achievable and likely to lead to sustained change was providing urgent training for staff, and then students, to improve awareness and knowledge, and to break the issue of the cycle of unaware students becoming unaware teachers.

Lack of diversity ‘breeds’ a lack of diversity. A lot of the main institutional barriers is the university’s lack of knowledge and the best way to deal with that is directly linked to how the under-represented students can like just you know break this barrier by teaching others and also by getting contact with the university.

Active bystander training was recommended to promote collective responsibility in challenging bias and negative views. Other suggestions included providing support for students from UrGs, countering negative views amongst peers and faculty, employing active strategies to promote patient diversity, being more equitable in services offered, and ensuring training was implemented. The final recommendation was to increase representativeness in the curriculum, as a way of training staff and students through regular exposure to up-to-date information regarding UrGs.

if the institutions were to be more aware [of EDI] and have [EDI training]…., I don’t know what training’s mandatory training’s given, but it would seem like potentially a lot of it [othering] could potentially be stopped. It just seems because you’ve got the lack of representation to faculty, race in faculty, they all sort of interlink with the other parts.

Participants felt that more and better training was needed for staff on EDI issues; a potential barrier to implementation was time, but short courses were expected to be effective.

every job I’ve ever done, either private sector, public sector, there is mandatory training and EDI’s, (…) human trafficking, (…) blackmail. (…) But I think we’re only talking like a half an hour.

Workshop results

Comments from nine workshop sessions (three each on student, staff and institutional EDI issues) were combined using frequency analysis to identify key themes and recommendations for change (Table  5 ). The strongest theme addressed stakeholders’ opinions about staff issues (96 comments in total), with recommendations about the need to improve staff attitudes [ 36 ], increase their awareness of students’ needs [ 15 ], and enhance communication skills [ 26 ]. The second main theme was student support [ 49 ], including the need to explore barriers to change [ 26 ] and improve access to support services [ 14 ]. Two other themes focused on the need to clarify and improve institutional EDI policies and processes [ 26 ] and ways to improve representation and diversity among student osteopaths, OEP staff and patients seeking osteopathic treatment [ 25 ] (also see Supplementary Material 5 ).

Overlapping themes were organised in Fig.  3 in relation to the groups involved in the recommended actions.

Workshop themes

The aims of this innovative mixed methods study were to survey student osteopaths’ levels of cultural humility to assess levels of awareness in the current educational environment and as a proxy for preparedness to work with patients from diverse backgrounds. It also explored the educational experiences of UrG students with the aim of improving equity, diversity and inclusivity (EDI) and sense of belonging in Osteopathic Educational Providers (OEPs). The survey response rate was 20%, but data was collected from 202 students from seven OEIs. 62 students identified with at least one UrG and 19 reported that they had been treated differently but 15 had not reported it.

Qualitative data from focus groups with students from the four selected UrGs suggested the main challenges faced were staff attitudes and lack of awareness; limited student support; and lack of representation in the curriculum and in institutional processes. These themes were explored and refined in interactive workshops, which generated recommendations to improve staff education, support students, and develop effective institutional policies. The implications of these findings are discussed below.

Educating staff

Cultural humility is a lifelong commitment to developing awareness to disparities experienced by people from diverse cultural groups, reflecting and being open to learning [ 49 , 50 , 51 , 52 ]. This model encourages practitioners to collaborate with patients, and educators to collaborate with students, to find solutions to discrimination and inequality based on their lived experiences and priorities [ 53 ]. Qualitative findings from the focus groups and workshops in this study indicated that experiences of ‘othering’ and discrimination were often associated with lack of cultural humility, self-awareness, ignorance, or overtly negative attitudes, mainly among staff. (Focus group theme 1: “ I don’t know if they can understand that it’s actually a knife that you’re throwing at someone and not just a joke ”).

There is limited evidence exploring the impact of cultural humility training with healthcare professional educators. Bakaa et al. [ 54 ] surveyed cultural competence in a sample of 3,000 chiropractors and reported similar findings which suggested that gaps between knowledge and self-reported behaviour required further research to clarify barriers and guide future training. Flateland et al. [ 55 ] concluded that inclusivity could be increased through mandatory diversity training which emphasised individual learning needs for students from all backgrounds and was supported by mentoring from personal academic tutors and a buddy system for UrG students.

A focus group study by Shapiro et al. [ 56 ] suggests that training increased awareness among third year medical students (first year of clinical training) but was less helpful in developing specific management skills. In contrast, another study found that, medical students tended to minimise the importance of self-awareness or the need to reflect on, and confront, personal biases [ 50 ]. Despite uncertainty about the impact of training, there is consensus that lack of training is also problematic. Whether based on concepts of cultural awareness, competence and humility [ 51 ] it is important that the sceptical perception that training is trying to be ‘politically correct’ is transformed into a way of rehumanising healthcare education [ 56 ]. Education in EDI and inclusive communication skills was strongly recommended by the participants in this study, but the challenges cited above suggest that ongoing monitoring would be needed to explore its’ impact on staff and students (Focus group theme 2: “ I don’t know what mandatory [EDI] training’s given, but it would seem like potentially a lot of it [othering] could potentially be stopped ”).

Supporting students

Inequalities in healthcare education are well documented [ 11 , 12 , 16 , 18 ]. Physiotherapy students from black, Asian and minority ethnic (BAME) backgrounds received lower marks in observed assessments compared to white students, with gaps in attainment also recorded for people with disabilities and students with non-traditional entry routes [ 10 ]. Overseas students, especially those who do not speak English as a first language, report isolation, loneliness, and lack of support, which is increased by intersectionality including race and gender [ 9 , 57 ]). In the survey, 15 students who felt they had been treated differently because of UrG characteristics did not report their difficulties, sometimes because they were unclear about whether an incident would count as discrimination or whether reporting a problem would have negative consequences (Workshop theme 3: “ Need to clarify what language/behaviour (e.g., ‘banter’) is acceptable ”).

Barriers to reporting misconduct include fear of not being believed, fear of repercussions and lack of confidence that complaints will be taken seriously [ 58 ]. Focus group and workshop comments suggested that students felt concerns were ignored, whether reported by individuals or year group representatives. The institution was rarely seen to take action to address the problems identified and there were concerns about consequences for people who spoke out. In contrast, some participants felt that whistleblowers should be valued and that incidents of discrimination could be reduced by encouraging more people to speak up (Workshop theme 4: “ Value all experiences and validate the ‘disruptor’ voice ”).

Research suggests that some of the factors that hinder the delivery of effective student support include limited disclosure of individual difficulties, especially for ‘invisible’ disabilities [ 18 ], the complex challenges faced by students with intersectional backgrounds [ 59 , 60 ], and lack of staff awareness, as discussed above [ 20 , 61 ]. Inconsistent institutional support practices also reinforce students’ disabled status and limit participation, rather than optimising their abilities and resilience [ 61 ], so there is a need to develop clear, robust systems to support students from UrGs, such as Active Bystander training (Workshop theme 2).

Improving institutional policies and processes

The practical processes used to support students and manage staff are grounded in an institution’s values and policies. Training inequalities are known to be a concern in medical and allied health professions and all HEIs in the UK have a responsibility to overcome the challenges of inaction in the face of discrimination. The General Medical Council has recently set new targets to eradicate disadvantage and discrimination in medical education and training [ 21 ]. Equality, diversity and inclusion pose challenges for small specialist universities, as noted in the ‘Changing the Culture’ (2016) framework, developed by Universities UK and GuildHE [ 62 ]. OEPs are expected to cultivate and maintain a culture of inclusion between staff, students and patients, train staff in EDI and ensure that staff are involved in the development of EDI policies [ 63 ]. This is reflected in the Quality Assurance Agency for Higher Education Subject Benchmark Statement for Osteopathy [ 64 ]: expectations and guidance on how OEPs can promote an EDI culture are provided. Participants in this study reported concerns about institutional knowledge (Focus group theme 2) and lack of clarity about how to access and use existing EDI policies (Focus group theme 1: “ How do I go about telling someone about that? ”).

In recent decades, access and participation from minority groups to higher education in the UK has been a core focus and entry rates for non-white students have increased: in 2019 they were higher for all ethnic groups compared with rates in 2006 and the entry rates increased in 2019 compared with 2018 [ 65 ]. There is limited information about experiences of inequalities reported by UrG students in osteopathic education or discrepancies in levels of attainment. A systematic review by MacMillan et al. [ 31 ] analysed discrimination, bullying and harassment in manual therapy education. They reported that there was evidence of widespread discrimination, harassment and bullying within manual therapy education; and there was a clear need for further research to focus upon the intersection of the characteristics identified as being linked to these experiences. Unfortunately, no osteopathic studies were found, although findings from physiotherapy and chiropractic education are likely to be transferable. Practising osteopaths from UrGs are also reported to be dissatisfied with lack of diversity within the profession and concerns have been raised about a lack of cultural competence training in OEPs [ 66 ].

Norris et al. [ 61 ] recommended that healthcare education institutions need to provide consistent and accessible information to help students find appropriate support and education to increase staff awareness about how individual experiences of disability affect learning. Complex EDI issues require university-wide approaches and AdvanceHE’s UK Equality Charter team proposes an ‘holistic approach’ [ 62 ]. Further research is needed to identify actions which would enhance educational experiences and outcomes for student osteopaths from UrGs. New data would also provide insights into the extent that osteopathic education prepares students to work with patients from UrGs and support long-term plans to enhance access and quality of patient care and attract more students from these UrGs to enhance the profession and represent more inclusively the communities they serve [ 31 , 64 ].

Limitations of the study

It is difficult to collect data from people who feel marginalised or vulnerable to discrimination, as demonstrated by low survey response rates with participants who typically have strong positive or negative views but few from the ‘silent majority’ (Shapiro et al. 2016). The MCHS is a new instrument which was adapted to osteopathy students, and due to the small sample size, it was not possible to get useful results with the confirmatory factor analysis. More research is also needed with this instrument to establish meaningful scores for dimensions of questionnaire. The response rate to this survey was low at 20% and there were fewer than 8 participants in all the focus groups. However, mixed designs enable compensation for some limitations of individual methods and data was collected from all seven UK OEPs. Two stages of qualitative analysis (focus groups and workshops) also enabled triangulation of the findings. The impact of facilitators as ‘insiders’ on data collection was not assessed and it was challenging to synthesise and weight results from the three stages.

Conclusions

The aims of this mixed methods study were to assess awareness of cultural humility among student osteopaths in the UK and to explore educational experiences of discrimination and ‘othering’ among students from underrepresented groups. Our findings are consistent with conclusions from other studies and the suggestions for action generated in workshops with diverse stakeholders are aligned with current EDI guidelines. Our three main recommendations are that OEIs prioritise actions to clarify institutional policies and processes to ensure they are accessible and effective in maintaining an inclusive educational environment; to review the adequacy of current student support services, particularly for underrepresented groups; and to provide EDI and communications skills training for staff to increase awareness about students’ learning needs and explore attitudinal barriers to change.

Data availability

The datasets generated and/or analysed during the current study are not publicly available due to the sensitivity of data collected and risk of identification of participants but are available from the corresponding author on reasonable request.

Szetela A. Black lives matter at five: limits and possibilities. Ethn Racial Stud. 2020;43(8):1358–83.

Article   Google Scholar  

Regulska J. The #MeToo movement as a global learning moment. Int High Educ. 2018;94:5–6.

Dodd V. Met police found to be institutionally racist, misogynistic and homophobic. The guardian [Internet]. 2023 Mar 21 [cited 2024 Jan 10]; https://www.theguardian.com/uk-news/2023/mar/21/metropolitan-police-institutionally-racist-misogynistic-homophobic-louise-casey-report .

Quinn B. People like you still uttered: BAME armed forces personnel on racism in services. The guardian [Internet]. 2020 Jan 7 [cited 2024 Jan 10]; https://www.theguardian.com/uk-news/2020/jan/07/people-like-you-still-uttered-bame-armed-forces-personnel-on-racism-in-services .

The report of the Commission on Race. and Ethnic Disparities [Internet]. Gov.uk. 2021 [cited 2024 Jan 10]. https://www.gov.uk/government/publications/the-report-of-the-commission-on-race-and-ethnic-disparities .

Adams R. Minority ethnic Britons’ educational success not reflected in pay. 2022.

Higher Education Statistics Agency. Higher Education Graduate Outcomes Statistics: UK, 2017/18 - Graduate activities and characteristics [Internet]. 2020 [cited 2024 Jan 10]. https://www.hesa.ac.uk/news/18-06-2020/sb257-higher-education-graduate-outcomes-statistics/activities .

Rimmer A. One in four trainee doctors report discriminatory behaviours from colleagues, GMC finds. BMJ. 2023;1601.

British Medical Association. Factors preventing progress of ethnic minority doctors into senior positions [Internet]. 2022 [cited 2024 Jan 10]. https://www.bma.org.uk/media/5748/bma-why-are-we-still-here-commissioned-report-june-15-2022.pdf .

Norris M, Hammond JA, Williams A, Grant R, Naylor S, Rozario C. Individual student characteristics and attainment in pre registration physiotherapy: a retrospective multi site cohort study. Physiotherapy. 2018;104(4):446–52.

Neves J, Hillman N. Student academic experience survey. H. E. P. I. a. H. E. Academy, editor.; 2020.

Bachmann CL, Gooch B. LGBT in Britain - universities Report. London: Stonewall; 2018.

Google Scholar  

Universities, and Colleges Admissions Service. Entry rates into higher education. Ethnicity, factsfigures [Internet]. 2023 [cited 2024 Jan 10]. https://www.ethnicity-facts-figures.service.gov.uk/education-skills-and-training/higher-education/entry-rates-into-higher-education/latest#by-ethnicity .

Johnson JL, Bottorff JL, Browne AJ, Grewal S, Hilton BA, Clarke H. Othering and being othered in the context of health care services. Health Commun. 2004;16(2):255–71.

Nixon SA. The coin model of privilege and critical allyship: implications for health. BMC Public Health. 2019;19(1):1637.

ESFA. Education and Skills agreements 2020 to 2021 [Internet]. Gov.uk. 2020 [cited 2024 Jan 10]. https://www.gov.uk/guidance/esfa-education-and-skills-agreements-2020-to-2021 .

Office for Students. Equality, diversity and student characteristics data - Office for Students. 2022 Jun 7 [cited 2024 Jan 10]; https://www.officeforstudents.org.uk/publications/equality-diversity-and-student-characteristics-data/?_cldee=Hgjd4C0mN-Ei1iAE9EzevenIhwSZ7Dxxo3wjIT8PIAo1SUA3S-H3H-VFCowOM6WI&recipientid=lead-46be407d3275ec1189430022481aa1a1-74564b23bb16438c9a14de1abd81377a&esid=cd365273-6de5-ec11-bb3c-002248004e75

UCAS. What is the experience of disabled students in education? [Internet]. 2022 [cited 2024 Jan 10]. https://www.ucas.com/file/610106/download?token=1kwt_gKE .

Hammond J, Marshall-Lucette S, Davies N, Ross F, Harris R. Spotlight on equality of employment opportunities: a qualitative study of job seeking experiences of graduating nurses and physiotherapists from black and minority ethnic backgrounds. Int J Nurs Stud. 2017;74:172–80.

Hammond JA, Williams A, Walker S, Norris M. Working hard to belong: a qualitative study exploring students from black, Asian and minority ethnic backgrounds experiences of pre-registration physiotherapy education. BMC Med Educ [Internet]. 2019;19(1). https://doi.org/10.1186/s12909-019-1821-6 .

General Medical Council. GMC targets elimination of disproportionate complaints and training inequalities [Internet]. 2021 [cited 2021 Autumn 5]. https://www.gmc-uk.org/news/news-archive/gmc-targets-elimination-of-disproportionate-complaints-and-training-inequalities .

Ross MH, Setchell J. People who identify as LGBTIQ + can experience assumptions, discomfort, some discrimination, and a lack of knowledge while attending physiotherapy: a survey. J Physiother. 2019;65(2):99–105.

Ross MH, Setchell J. Physiotherapists vary in their knowledge of and approach to working with patients who are LGBTQIA+: a qualitative study. J Physiother. 2023;69(2):114–22.

Council of Deans. Anti-racism in AHP Education: Building an Inclusive Environment [Internet]. 2023 [cited 2024 Jan 10]. https://www.councilofdeans.org.uk/wp-content/uploads/2023/03/Anti.racism.in_.ahp_.education.report.pdf .

House of Commons. House of Commons Women and Equalities Committee Black maternal health Third Report of. Session 2022–23 Report, together with formal minutes relating to the report Ordered by the House of Commons [Internet]. 2023 [cited 2024 Jan 10]. https://committees.parliament.uk/publications/38989/documents/191706/default/ .

Gonzales G, Przedworski J. Comparison of health and health risk factors between lesbian, gay, and bisexual adults and heterosexual adults in the United States: Results from the National Health…. JAMA Intern Med [Internet]. 2016; https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2530417?casa_token=CRF1uNk3F_8AAAAA:pJQ_7UypXiwhMMO0KeqLEGf5Jb6fZC_EBg0QrmXRF507vY5Pl7srcTta6fWsDqHmuOQCZYIGoQ&casa_token=OWWuxn8UEDwAAAAA:4YRNKNxWmG-6LZfnl_MEATduHjHiawDrLprat8Y59Nopll1K-Yw1HV-zL5K15z5lZBn1X9FPUg

Cross TL. Towards a culturally competent system of care: a monograph on effective services for minority. Washington, DC: Georgetown University; 1989.

Hook JN, Davis DE, Owen J, Worthington EL, Utsey SO. Cultural humility: measuring openness to culturally diverse clients. J Couns Psychol. 2013;60(3):353–66.

NHS England. Chief Allied Health Professions Officer extends her remit to two additional professions [Internet]. 2017 [cited 2024 Jan 10]. https://www.england.nhs.uk/2017/04/chief-allied-health-professions-officer-extends-her-remit-to-two-additional-professions/ .

General Osteopathic Council. Standards of Practice [Internet]. 2023 [cited 2024 Jan 10]. https://www.osteopathy.org.uk/standards/ .

MacMillan A, Hohenschurz-Schmidt D, Migliarini DV, Draper-Rodi DJ. Discrimination, bullying or harassment in undergraduate education in the osteopathic, chiropractic and physiotherapy professions: a systematic review with critical interpretive synthesis. Int J Educ Res Open. 2022;3(100105):100105.

O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13(2):92–8.

Creswell JW, Clark VP. Designing and conducting mixed methods research. SAGE; 2018.

Mertens DM. Transformative paradigm. J Mix Methods Res. 2007;1(3):212–25.

Creswell JW. Research Design: qualitative, quantitative, and mixed methods approaches. SAGE; 2014.

Ball S, Harshfield S, Carpenter A, Bertscher A, Marjanovic S. Patient and public involvement and engagement in research. Santa Monica: RAND Corporation; 2019.

Gonzalez E, Sperandio KR, Mullen PR, Tuazon VE. Development and initial testing of the multidimensional cultural humility scale. Meas Eval Couns Dev. 2021;54(1):56–70.

Finch H. Focus groups. Qualitative research practice: A guide for social science students and researchers. Ritchie J, London LJ, editors. J Ritchie and L J. 2003;170–98.

Holloway I, Wheeler S. Qualitative research in nursing and healthcare. Wiley; 2010.

Kite J, Phongsavan P. Insights for conducting real-time focus groups online using a web conferencing service. F1000Res. 2017;6:122.

Mertens DM. Transformative mixed methods research. Qual Inq. 2010;16(6):469–74.

Kaiser K. Protecting respondent confidentiality in qualitative research. Qual Health Res. 2009;19(11):1632–41.

R Core Team. A Language and Environment for Statistical Computing [Internet]. R Foundation for Statistical Computing, Vienna, Austri; 2023 [cited 2024 Oct 1]. https://www.R-project.org/ .

Rosseel Y. lavaan: AnRPackage for Structural Equation Modeling. J Stat Softw [Internet]. 2012;48(2). https://doi.org/10.18637/jss.v048.i02 .

Van Buuren S. Groothuis-Oudshoorn, mice: Multivariate imputation by chained equations in r. J Stat Softw. 2011;45:1–67.

Welch BL. The generalization of ‘student’s’ problem when several different population varlances are involved. Biometrika. 1947;34(1–2):28–35.

Braun V, Clarke V. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health. 2019;11(4):589–97.

Yvonne Feilzer M. Doing mixed methods research pragmatically: implications for the rediscovery of pragmatism as a research paradigm. J Mix Methods Res. 2010;4(1):6–16.

Tollemache N, Shrewsbury D, Llewellyn C. Que(e) rying undergraduate medical curricula: a cross-sectional online survey of lesbian, gay, bisexual, transgender, and queer content inclusion in UK undergraduate medical education. BMC Med Educ [Internet]. 2021;21(1). https://doi.org/10.1186/s12909-021-02532-y .

Shepherd SM. Cultural awareness workshops: limitations and practical consequences. BMC Med Educ. 2019;19(1):14.

Tervalon M, Murray-García J. Cultural humility versus cultural competence: a critical distinction in defining physician training outcomes in multicultural education. J Health Care Poor Underserved. 1998;9(2):117–25.

Foronda C, Baptiste D-L, Reinholdt MM, Ousman K. Cultural humility. J Transcult Nurs. 2016;27(3):210–7.

Solchanyk D, Ekeh O, Saffran L, Burnett-Zeigler IE, Doobay-Persaud A. Integrating cultural humility into the medical education curriculum: strategies for educators. Teach Learn Med. 2021;33(5):554–60.

Bakaa N, Southerst D, Côté P, Macedo L, Carlesso LC, MacDermid J et al. Assessing cultural competency among Canadian chiropractors: a cross-sectional survey of Canadian Chiropractic Association members. Chiropr Man Therap [Internet]. 2023;31(1). https://doi.org/10.1186/s12998-023-00474-4 .

Flateland SM, Pryce-Miller M, Skisland AV-S, Tønsberg AF, Söderhamn U. Exploring the experiences of being an ethnic minority student within undergraduate nurse education: a qualitative study. BMC Nurs [Internet]. 2019;18(1). https://doi.org/10.1186/s12912-019-0389-0 .

Jowsey T. Three zones of cultural competency: surface competency, bias twilight, and the confronting midnight zone. BMC Med Educ [Internet]. 2019;19(1). https://doi.org/10.1186/s12909-019-1746-0 .

Shapiro J, Lie D, Gutierrez D, Zhuang G. That never would have occurred to me: a qualitative study of medical students’ views of a cultural competence curriculum. BMC Med Educ [Internet]. 2006;6(1). https://doi.org/10.1186/1472-6920-6-31 .

Equality. and Human Rights Commission. Tackling racial harassment: Universities challenged. 2019.

Gottlieb M. The case for a cultural humility framework in social work practice. J Ethn Cult Divers Soc Work. 2021;30(6):463–81.

LeBlanc C, Sonnenberg LK, King S, Busari J. Medical education leadership: from diversity to inclusivity. GMS J Med Educ. 2020;37(2):Doc18.

Norris M, Hammond J, Williams A, Walker S. Students with specific learning disabilities experiences of pre-registration physiotherapy education: a qualitative study. BMC Med Educ [Internet]. 2020;20(1). https://doi.org/10.1186/s12909-019-1913-3 .

GuildHE. EDI & ANTI-RACISM BRIEFING #15. Tackling Harassment and the importance of EDI. 2021.

General Osteopathic Council. Graduate Outcomes and Standards for Education and Training [Internet]. London. 2022. https://www.osteopathy.org.uk/news-and-resources/document-library/publications/graduate-outcomes-and-standards-for-education-and-training/ .

Quality Assurance Agency for Higher Education. Subject Benchmark Statement Osteopathy. Gloucester: QAA; 2024.

Universities, and Colleges Admissions Service. Entry rates into higher education [Internet]. Entry rates into higher education. Ethnicity, factsfigures. 2020 [cited 2021 Aug 2]. https://www.ethnicity-facts-figures.service.gov.uk/education-skills-and-training/higher-education/entry-rates-into-higher-education/latest#by-ethnicity .

MacMillan A, Corser A, Clark Z. Inclusivity and accessibility in undergraduate osteopathic education for students with disability: a scoping review. Int J Osteopath Med. 2021;40:38–45.

Download references

Acknowledgements

Our thanks to Dr Phil Bright and Mr Dévan Rajendran for their support with the project, including the community engagement meetings.

This project received funding from four organisations: The Osteopathic Foundation provided £20,000, the General Osteopathic Council provided £7,500, the University College of Osteopathy provided £7,500, and the Institute of Osteopathy provided £3,000. The authors, including the Principal Investigator, are employed by the University College of Osteopathy. However, the University College of Osteopathy and other funders did not have any specific role in the conceptualization, design, data collection, analysis, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

University College of Osteopathy, 275 Borough High Street, SE1 1JE, London, UK

Jerry Draper-Rodi, Hilary Abbey, Oliver T. Thomson, Kevin Brownhill, Andrew MacMillan, Yinka Fabusuyi & Steven Vogel

National Council for Osteopathic Research, 275 Borough High Street, SE1 1JE, London, UK

Jerry Draper-Rodi

School of Allied and Public Health Professions, North Holmes Road, CT1 1QU, Canterbury, Kent, UK

John Hammond

University of Portsmouth, University House, Winston Churchill Ave, PO1 2UP, Hampshire, Portsmouth, England

Andrew MacMillan

You can also search for this author in PubMed   Google Scholar

Contributions

JDR, HA and SV designed the study and applied for ethical approval. AMM, HA, JDR, SV, YF facilitated the focus groups. KB collected and analysed the quantitative data; OT analysed and interpreted the qualitative data. HA, JDR, SV, YF facilitated the workshops. HA and JDR analysed the workshop data. All authors contributed, read and approved the final manuscript.

Corresponding author

Correspondence to Jerry Draper-Rodi .

Ethics declarations

Ethics approval and consent to participate.

The study was approved by the University College of Osteopathy Research Ethics Committee. For the survey, consent was assumed by return of the completed questionnaire (this was explained in the participant information sheet), and for the focus groups, informed consent was obtained from all participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Draper-Rodi, J., Abbey, H., Hammond, J. et al. Overcoming barriers to equality, diversity, inclusivity, and sense of belonging in healthcare education: the Underrepresented Groups’ Experiences in Osteopathic Training (UrGEnT) mixed methods study. BMC Med Educ 24 , 468 (2024). https://doi.org/10.1186/s12909-024-05404-3

Download citation

Received : 02 February 2024

Accepted : 09 April 2024

Published : 26 April 2024

DOI : https://doi.org/10.1186/s12909-024-05404-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Underrepresented
• Social Justice
• Equity diversity and inclusion
• Transformative action research
• Osteopathic undergraduate education
• Ethnic minorities

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Study Protocol
• Open access
• Published: 22 April 2024

Implementation and early effects of medicaid policy interventions to promote racial equity in pregnancy and early childhood outcomes in Pennsylvania: protocol for a mixed methods study

• Marian Jarlenski 1 ,
• Evan Cole 1 ,
• Christine McClure 1 ,
• Sarah Sanders 2 ,
• Marquita Smalls 2 &
• Dara D Méndez 2  

BMC Health Services Research volume  24 , Article number:  498 ( 2024 ) Cite this article

127 Accesses

1 Altmetric

Metrics details

There are large racial inequities in pregnancy and early childhood health within state Medicaid programs in the United States. To date, few Medicaid policy interventions have explicitly focused on improving health in Black populations. Pennsylvania Medicaid has adopted two policy interventions to incentivize racial health equity in managed care (equity payment program) and obstetric service delivery (equity focused obstetric bundle). Our research team will conduct a mixed-methods study to investigate the implementation and early effects of these two policy interventions on pregnancy and infant health equity.

Qualitative interviews will be conducted with Medicaid managed care administrators and obstetric and pediatric providers, and focus groups will be conducted among Medicaid beneficiaries. Quantitative data on healthcare utilization, healthcare quality, and health outcomes among pregnant and parenting people will be extracted from administrative Medicaid healthcare data. Primary outcomes are stakeholder perspectives on policy intervention implementation (qualitative) and timely prenatal care, pregnancy and birth outcomes, and well-child visits (quantitative). Template analysis methods will be applied to qualitative data. Quantitative analyses will use an interrupted time series design to examine changes over time in outcomes among Black people, relative to people of other races, before and after adoption of the Pennsylvania Medicaid equity-focused policy interventions.

Findings from this study are expected to advance knowledge about how Medicaid programs can best implement policy interventions to promote racial equity in pregnancy and early childhood health.

Peer Review reports

Rates of maternal and infant morbidity and mortality in the United States far exceed those of comparable nations [ 1 ]. The burdens of racist policies have produced vastly worse outcomes for Black and Native, relative to White, populations [ 2 ]. For example, Black and Native birthing people are more than three times as likely to experience pregnancy-related mortality compared to white birthing people [ 3 ]. For every pregnancy-related death, there are thousands of birthing people who experience severe morbidity; including stark racial disparities where Black populations are more likely to experience stroke or serious cardiovascular events sending them on a trajectory of adverse health outcomes beyond pregnancy [ 4 , 5 ]. We also see similar racial inequities for infant mortality and morbidity. These racial inequities are not adequately explained by individual behaviors or other socio-economic factors, but are a complex intersection of factors shaped by structural and social determinants [ 2 , 6 ], policies and institutions carrying out such policies [ 7 ]. There is a long history of structural racism that has resulted in practices and policies that have had a detrimental impact on Black and Indigenous populations in the United States [ 8 ].

State Medicaid programs are the largest single payer for pregnancy and birth in the US, covering 68% of births to Black people [ 9 ]. As such, Medicaid programs have great potential to implement structural interventions to advance racial equity in healthcare and health outcomes during pregnancy and postpartum [ 10 ]. Historically, Medicaid policies have promoted equality, that is, providing equal benefits to all regardless of the distribution of need [ 11 ]. An equity-focused policy approach, however, will direct resources toward improving health and well-being among those with the greatest need [ 12 ]. Unfortunately, a vast body of research conducted among Medicaid-enrolled populations shows that healthcare systems do not provide the same quality of obstetric care to Black people and other people of color, relative to white people [ 13 , 14 , 15 , 16 , 17 , 18 ].

Pennsylvania’s Medicaid program is the fourth-largest in the United States, with 3.5 million people enrolled and expenditures of $35.1 billion [ 19 , 20 ]. Past research in the Pennsylvania Medicaid program has demonstrated Black people were less able to access prenatal and postpartum care relative to those in other race groups [ 15 ]. Reporting from the Pennsylvania Maternal Mortality Commission shows that in more than half of the cases of pregnancy-associated deaths, the decadents were enrolled in Medicaid [ 21 ]. Similar to national figures, pregnancy-associated death was far more common among Black people vs. those of other races ( [ 21 ].

To ameliorate these racial disparities, Pennsylvania Medicaid is currently implementing two novel policies with the goal to advance racial equity in pregnancy and child health. The first, the equity incentive payment program, was initiated in 2020. The equity incentive payment program makes available approximately $26 million in Medicaid managed care organization (MCO) payments each year to plans that improve access to timely prenatal care and well-child visits among Black beneficiaries. The second is the maternity care bundled payment model, initiated in 2021, designed to provide incentives to obstetric providers across a wide range of pregnancy health outcomes and specifically incentivizes improvements among Black beneficiaries.

Although these policy approaches are unique, it is possible that other state Medicaid programs or other health insurers could learn from the policies and adapt or expand these approaches. Our research team will conduct a mixed-methods study to investigate the implementation and early effects of the two aforementioned policy changes on pregnancy and infant health equity. Our research aims are to: (1) evaluate implementation and early effects of the equity incentive payment program prenatal and early childhood healthcare outcomes and experiences among Black Medicaid beneficiaries; and (2) determine the extent to which an equity-focused maternity care bundled payment model affects racial equity in obstetric care and pregnancy health outcomes. To achieve these aims, we will draw on established partnerships between university researchers, community organizations, and policymakers to collect and analyze data. First, we will collect qualitative data with diverse stakeholders including Medicaid beneficiaries, MCO plan representatives, and pediatric and obstetric care clinicians to study implementation of these equity-focused policy changes. Second, we will use a community-partnered approach to develop a quantitative analysis plan of Medicaid administrative data for an estimated 167,000 birthing person-child dyads to estimate early effects of these policies. Our cross-disciplinary, community-engaged partnerships will enable us to triangulate how the healthcare policy structures of state Medicaid programs can be changed to promote racial equity in health.

Theoretical framework

The proposed research seeks to advance knowledge about the causes of, and structural interventions to improve, health and well-being among Black pregnant and parenting persons and their children in Medicaid. The theoretical model underlying this work is informed by foundational literature from a range of disciplines. First, it incorporates Critical Race Theory and Public Health Critical Race Praxis, which posit structural determinants, such as racism and other forms of oppression (e.g., sexism, classism, poverty), as fundamental causes of adverse social environments that interact to make certain populations more susceptible to illness and resulting in suboptimal health [ 22 , 23 , 24 , 25 , 26 ]. Second, it incorporates political science theory that dominant social definitions of populations shape group empowerment and resulting health policies and material benefits [ 27 ]. Third, it draws on new scholarship suggesting the necessity of studying social welfare policies with a critical race lens centering the beneficiaries’ lived experiences [ 11 , 28 , 29 ].

As depicted in Fig.  1 , our research project identifies both the Medicaid policy environment as well as the beneficiary experiences of the policy environment as upstream factors that influence healthcare organization and beneficiaries’ interaction with healthcare systems. In particular, we aim to facilitate and further enhance the connection between beneficiaries’ lived experiences and policy decision-makers through our collaboration with community partners. This connection can influence the policymaking process that shapes how care is delivered both at the managed care and healthcare provider levels. Healthcare utilization and quality are conceptualized as intermediate outcomes which may influence pregnancy and birth outcomes.

Conceptual model of the evaluation of structural interventions in Medicaid to promote racial equity in pregnancy and child health

Medicaid policy interventions

Nearly all Medicaid beneficiaries in Pennsylvania are enrolled in 1 of 8 Medicaid managed care plans, which manage the physical health care of enrollees and are subject to pay-for-performance requirements for healthcare quality measures. Currently, the Pennsylvania Medicaid program makes available 2% of total payments to MCO plans, contingent on MCO plan performance on 13 different healthcare quality metrics. An equity incentive payment program was added to this reimbursement scheme for two metrics in 2020: timely prenatal care and well-child visit utilization in the first 15 months of life (Fig.  2 ). Specifically, 2/13 (or 0.15%) of total payments are withheld for these two equity-focused metrics. MCO plans are assessed on overall performance and subsequently on the annual improvement on these measures among Black beneficiaries. MCO plans can be penalized (up to -0.12% of total payments) or rewarded (up to + 0.35% of total payments) for their performance on each of these two metrics.

Pennsylvania Medicaid’s health equity incentive payment program will condition payments to managed care organizations based on overall performance as well as improvement among Black beneficiaries

Second, Pennsylvania Medicaid implemented a maternity care bundled payment model in 2021 that considers outcomes among Black beneficiaries (Fig.  3 ). Under maternity care bundled payment models, obstetric providers are incentivized to meet a total cost threshold and quality metrics for prenatal and delivery care [ 30 ]. Specifically, providers and payers agree on a target cost for low- or average-risk perinatal care, including pregnancy, delivery, and postpartum care. If total payments to providers are lower than the target cost while maintaining certain quality metrics, providers and payers share those savings. Under Pennsylvania’s new model, providers are able to achieve shared savings based on quality metric performance, as well as a health equity score reflecting performance on those metrics among Black beneficiaries.

Pennsylvania Medicaid’s equity-focused maternity bundled payment model will allow for shared savings between obstetric care providers and managed care organizations, allowing for extra shared savings among providers whose Black patients experience better outcomes

Qualitative data Collection

To understand the interventions and responses to these policies, as well as related implementation barriers and facilitators, we will conduct interviews with each at least two representatives from each MCO ( n  = 18). We will partner with colleagues from the Department of Human Services (DHS) to identify relevant MCO representatives. Interviews will elucidate MCOs’ perspectives, processes used by MCOs to design their interventions (e.g., review of existing evidence, input from community members or providers who serve them), anticipated effects, and sustainability of these payment policy changes. The goal is for some of the results of these interviews to inform our understanding of the implementation process which will be further explored in the interviews and focus groups with clinicians and Medicaid recipients.

In collaboration with the Community Health Advocates (CHA) program led by Healthy Start Pittsburgh, as well as other community and organizational partners across the state, we will recruit current and former Medicaid beneficiaries for focus group participation. We aim to recruit  ∼  50 community participants and will purposively oversample Black participants and will aim to recruit people of all ethnicities who identify as Black and multi-racial in order to achieve our aims of elucidating the experiences of Black parenting and pregnant people in Medicaid. Inclusion criteria are: current pregnancy or pregnant within the past 2 years; current or former enrollment in Pennsylvania Medicaid and/or Healthy Start; and ability to complete the interview in English.

Finally, we will partner with colleagues from DHS to identify pediatric and obstetric health professionals for interviews regarding the maternity bundled payment program and key outcomes related to the equity incentive payment. We will also use Medicaid administrative data to identify providers who serve Black beneficiaries and invite them to participate. We will aim to interview at least 20 obstetric and pediatric healthcare professionals to elucidate their perspectives on how structural racism in medicine affects patient outcomes, and the types of Medicaid policy changes that should be implemented.

We developed separate focus group/interview guides for community members, MCO leaders, and healthcare professionals. Each interview guide consists of open-ended questions to elucidate participants’ experiences with Medicaid; desired policy changes in Medicaid (among beneficiary participants); perceived steps that would be useful to combat anti-Black racism in healthcare and social services (especially among Black participants); and perspectives about the new Medicaid policies. Additionally, the interview guides ask demographic questions regarding gender identity, race, and ethnicity. We will first pilot-test the guide with our research partners and Healthy Start CHAs for clarity of question wording. All interviews will take place in-person in a private office space, or over the phone or videoconference, according to participants’ preferences and COVID-19 protocols. The interviewer will describe study objectives to each participant, obtain consent, and each interview will be audio-recorded and the interviewer will take notes throughout. Interview audio recordings will be transcribed verbatim, and transcripts spot-checked against the audio recordings for accuracy. The audio recording files will then be deleted to protect confidentiality of participants.

Qualitative data analysis

Study data will be analyzed and reported using the Consolidated Criteria for Reporting Qualitative Research (COREQ) Framework [ 31 ]. To analyze data, we will use template analysis, which combines features of deductive content analysis and inductive grounded theory, thereby allowing us to obtain specific information while also capturing any new or unanticipated themes [ 32 ]. Two coders will separately code the first 3 interview transcripts, meet to compare codes, discuss inconsistency in coding approaches, and then alter or add codes. This iterative process will be repeated until the coding scheme is fully developed. The coders will independently code all transcripts, and any coding discrepancies will be resolved via discussion. Once coding is complete, synthesis of content will begin by organizing codes under broader domains (meta-codes) as well as sub-codes. The primary analysis will be to convey qualitative data, including the use of illustrative quotes.

Quantitative healthcare data and analysis

Administrative healthcare data from the Pennsylvania Medicaid program, with linked birthing person-child dyads, will be used to create our quantitative analytic data. Medicaid data include a census of enrollment, hospital, outpatient/professional, pharmaceutical, and provider data for all beneficiaries in the Pennsylvania Medicaid program. Importantly, data contain self-reported race and ethnicity that is provided at the time of Medicaid enrollment (< 2% missing); as well as time-varying data on 9-digit ZIP code of residence. Data also include the amounts paid from Medicaid MCOs to healthcare providers for all medical services. Table  1 shows baseline data from Pennsylvania Medicaid-enrolled persons with a livebirth delivery in 2019, overall and stratified by race of the birthing person. We will also conduct sensitivity analyses to examine dyads that are multi-racial.

We will employ a comparative interrupted time series (ITS) analyses with a nonequivalent comparison group to estimate policy effects. Specifically, we will evaluate: (1) the extent to which the equity incentive payment program improved timely prenatal care and well-child visits among Black beneficiaries, relative to those of other races; and (2) the extent to which healthcare provider participation in the equity-focused maternity bundled payment model improved healthcare and health outcomes among Black beneficiaries, relative to those of other races.

For Aim 1, outcomes include binary measures of initiating prenatal care in the first trimester, and children receiving at least 6 well-child visits in the first six months of life. We will compare outcomes among Black beneficiaries relative to those of other racial groups, post- relative to pre- implementation of the equity incentive payment program. For Aim 2, outcomes include a composite of prenatal care quality measures (social determinants of health screening, prenatal and postpartum depression screening and follow-up, immunization, screening and treatment for substance use disorders, postpartum visit attendance), gestational age and birthweight, and severe maternal morbidity [ 33 ]. For both aims, multivariable regression models will control for maternal age, ethnicity, parity, ZIP code of residence, MCO plan enrollment, Medicaid eligibility category (expansion, pregnancy, disability, or others), and indices of obstetric and pediatric clinical comorbidities [ 34 , 35 ].

Sensitivity analyses

Analyses are designed to estimate early effects of the policies and should be interpreted alongside the qualitative results regarding policy implementation and beneficiary experiences. One assumption of ITS analyses is that our comparison groups approximate a counterfactual scenario for the intervention groups [ 36 , 37 , 38 ]. Although trends in Black-White inequities in pregnancy and child outcomes have, unfortunately, persisted over time [ 39 ], the COVID-19 pandemic has differentially burdened Black and Latina/x people relative to other race and ethnic groups [ 40 , 41 ]. Effects of the pandemic on pregnancy outcomes are only just beginning of what is to be explored [ 42 ]. It is therefore possible that we will not be able to disentangle policy effects from effects of COVID-19. To address this limitation, we will employ area-level rates of COVID-19 infection as control variables and for Aim 1 (equity incentive payment) we will conduct a sub-analysis investigating trends in 2021 vs. 2020. We chose to evaluate outcomes for Aim 2 (maternity care bundled payment) only in 2021, comparing the statistical intervention of race*provider. Finally, our qualitative analyses will provide context on differential impacts of COVID-19, which will inform interpretation of the quantitative results.

This study was approved by the University of Pittsburgh Institutional Review Board (Study # 23090108).

This mixed-methods research will investigate the extent to which changes in the Pennsylvania Medicaid program are associated with improvements in access to medical care and health outcomes among Black pregnant and birthing persons and their children. Our past research found that Black childbearing people in Pennsylvania Medicaid consistently experienced worse healthcare and health outcomes, compared to those of other racial and ethnic groups [ 43 , 44 ]. Racism in healthcare and other systems manifests in systematically worse access to and quality of care and other services for Black childbearing people [ 8 ]. In addition to suboptimal healthcare experiences, historical policies and practices such as residential redlining and segregation have resulted in lower wealth attainment in Black communities resulting in inequities in neighborhood factors and perinatal health [ 45 , 46 , 47 ].

The policies under study involve modifying common Medicaid reimbursement arrangements– namely, pay-for-performance programs and maternity care bundled payments. The policies are modified to embed financial incentives for Medicaid health plans and healthcare providers to improve the quality of care and health outcomes for Black pregnant and parenting persons and their children. These are the first such payment policies, to our knowledge, that explicitly aim to promote racial health equity with an explicit focus on addressing inequities that affect Black and Indigenous populations in Pennsylvania.

Interest from policymakers in payment reforms to improve health equity has increased recently; however, information on the implementation and effects of such models is sparse [ 48 , 49 ]. Although several state Medicaid programs have adopted maternity care bundled payment models, prior models have not considered racial inequities in pregnancy outcomes [ 30 , 50 ]. In 2012, Oregon adopted regional health equity coalitions as part of the state Medicaid program’s transition to Coordinated Care Organizations (CCOs). CCOs were required and given support to develop strategies that would address racial health disparities within the Medicaid population, and the regional health equity coalitions included underrepresented stakeholders to guide CCOs in the development of these interventions. While CCOs did reduce Black-white differences in primary care utilization and access to care within 3 years of policy implementation, it did not impact disparities in emergency department utilization [ 51 ]. The current research project will add to the extant evidence on how Medicaid programs can use policy to promote racial health equity.

Our study is limited in investigating the direct effects of the pandemic on racial inequities in perinatal and infant health and the intersections between the effects of the pandemic and the effects of the aforementioned Medicaid policies. However, we will have the ability to look at changes in outcomes over time. Additionally, these payment reform interventions focus largely on transforming the financing and delivery of healthcare, drawing attention to the structural and social determinants of health in the healthcare system. It is estimated that medical care contributes 10–20% to health outcomes; health and well-being are also shaped by factors such as environmental and socioeconomic conditions [ 52 ].

This study will contribute to the current body of knowledge about the recent interventions in Medicaid focused on racial equity. Specifically, findings will shed light on how the equity-focused obstetric care policies are being implemented and provide an evaluation of effects on health outcomes. These results can be used for future adaptions of the policy interventions or to inform the adoption of such equity-focused policies in different geographic regions of the United States.

Data availability

No datasets were generated or analysed during the current protocol study.

Abbreviations

Managed Care Organization

Community Health Advocate

Coordinated Care Organization

Alkema L, Chou D, Hogan D, Zhang S, Moller AB, Gemmill A, et al. Global, regional, and national levels and trends in maternal mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN Maternal Mortality Estimation Inter-agency Group. Lancet. 2016;387(10017):462–74.

Article   PubMed   Google Scholar  

Wang E, Glazer KB, Howell EA, Janevic TM. Social determinants of pregnancy-related mortality and morbidity in the United States: a systematic review. Obstet Gynecol. 2020;135(4):896–915.

Article   PubMed   PubMed Central   Google Scholar  

Hoyert D. Maternal mortality rates in the United States, 2020. NCHS Health E-Stats; 2022.

Fingar K, Hambrick MM, Heslin KC, Moore JE. Trends and Disparities in Delivery Hospitalizations Involving Severe Maternal Morbidity, 2006–2015. Rockville, MD. 2018.

Thomas SB, Quinn SC, Butler J, Fryer CS, Garza MA. Toward a fourth generation of disparities research to achieve health equity. Annu Rev Public Health. 2011;32:399–416.

Crear-Perry J, Correa-de-Araujo R, Lewis Johnson T, McLemore MR, Neilson E, Wallace M. Social and Structural Determinants of Health Inequities in maternal health. J Womens Health (Larchmt). 2021;30(2):230–5.

Jones CP. Levels of racism: a theoretic framework and a gardener’s tale. Am J Public Health. 2000;90(8):1212–5.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Prather C, Fuller TR, Jeffries WLt, Marshall KJ, Howell AV, Belyue-Umole A, et al. Racism, African American Women, and their sexual and Reproductive Health: a review of historical and Contemporary Evidence and Implications for Health Equity. Health Equity. 2018;2(1):249–59.

Markus AR, Andres E, West KD, Garro N, Pellegrini C. Medicaid covered births, 2008 through 2010, in the context of the implementation of health reform. Womens Health Issues. 2013;23(5):e273–80.

Headen IE, Elovitz MA, Battarbee AN, Lo JO, Debbink MP. Racism and perinatal health inequities research: where we have been and where we should go. Am J Obstet Gynecol. 2022.

Michener JSM, Thurston C. From the Margins to the Center: A Bottom- Up Approach to Welfare State Scholarship. Perspectives on Politics. 2020:1–16.

Braveman PAE, Orleans T, Proctor D, Plough A. What is health equity? And what difference does a definition make? Princeton, NJ: Robert Wood Johnson Foundation; 2017.

Google Scholar  

Thiel de Bocanegra H, Braughton M, Bradsberry M, Howell M, Logan J, Schwarz EB. Racial and ethnic disparities in postpartum care and contraception in California’s Medicaid program. Am J Obstet Gynecol. 2017;217(1):47. e1- e7.

Article   Google Scholar  

Gao YA, Drake C, Krans EE, Chen Q, Jarlenski MP. Explaining racial-ethnic disparities in the receipt of medication for opioid use disorder during pregnancy. J Addict Med. 2022.

Parekh N, Jarlenski M, Kelley D. Prenatal and Postpartum Care Disparities in a large Medicaid Program. Matern Child Health J. 2018;22(3):429–37.

Gavin NI, Adams EK, Hartmann KE, Benedict MB, Chireau M. Racial and ethnic disparities in the use of pregnancy-related health care among Medicaid pregnant women. Matern Child Health J. 2004;8(3):113–26.

Aseltine RH Jr., Yan J, Fleischman S, Katz M, DeFrancesco M. Racial and ethnic disparities in Hospital readmissions after Delivery. Obstet Gynecol. 2015;126(5):1040–7.

Wagner JL, White RS, Tangel V, Gupta S, Pick JS. Socioeconomic, racial, and ethnic disparities in Postpartum readmissions in patients with Preeclampsia: a multi-state analysis, 2007–2014. J Racial Ethn Health Disparities. 2019;6(4):806–20.

Kaiser Family Foundation. Total Monthly Medicaid/CHIP Enrollment and Pre-ACA Enrollment: Kaiser Family Foundation; 2022 [ https://www.kff.org/health-reform/state-indicator/total-monthly-medicaid-and-chip-enrollment/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D .

Kaiser Family Foundation, Total Medicaid Spending. FY 2020 2022 [ https://www.kff.org/medicaid/state-indicator/total-medicaid-spending/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D .

Pennsylvania Department of Health. Pregnancy-Associated Deaths in Pennsylvania, 2013–2018. Harrisburg, PA. 2020 December 2020.

Geronimus AT, Hicken MT, Pearson JA, Seashols SJ, Brown KL, Cruz TD. Do US Black women experience stress-related Accelerated Biological Aging? A Novel Theory and First Population-based test of black-white differences in telomere length. Hum Nat. 2010;21(1):19–38.

Marmot M. Health in an unequal world. Lancet. 2006;368(9552):2081–94.

Ekeke P, Mendez DD, Yanowitz TD, Catov JM. Racial differences in the biochemical effects of stress in pregnancy. Int J Environ Res Public Health. 2020;17:19.

Ford CL, Airhihenbuwa CO. Critical race theory, race equity, and public health: toward antiracism praxis. Am J Public Health. 2010;100(Suppl 1):S30–5.

Bailey ZD, Krieger N, Agenor M, Graves J, Linos N, Bassett MT. Structural racism and health inequities in the USA: evidence and interventions. Lancet. 2017;389(10077):1453–63.

Schneider AIH. Social Construction of Target populations: implications for politics and policy. Am Polit Sci Rev. 1993;87(2):334–47.

Michener J. Fragmented democracy: medicaid, federalism, and unequal politics. Cambridge, United Kingdom; New York, NY, USA: Cambridge University Press; 2018. xii, 226 pages p.

Ford CL, Airhihenbuwa CO. The public health critical race methodology: praxis for antiracism research. Soc Sci Med. 2010;71(8):1390–8.

Carroll C, Chernew M, Fendrick AM, Thompson J, Rose S. Effects of episode-based payment on health care spending and utilization: evidence from perinatal care in Arkansas. J Health Econ. 2018;61:47–62.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Centers for Disease Control and Prevention. Severe Maternal Morbidity in the United States 2021 [ http://www.cdc.gov/reproductivehealth/maternalinfanthealth/severematernalmorbidity.html .

Bateman BT, Mhyre JM, Hernandez-Diaz S, Huybrechts KF, Fischer MA, Creanga AA, et al. Development of a comorbidity index for use in obstetric patients. Obstet Gynecol. 2013;122(5):957–65.

Feudtner C, Feinstein JA, Zhong W, Hall M, Dai D. Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation. BMC Pediatr. 2014;14:199.

Ashenfelter OCD. Using the longitudinal structure of earnings to estimate the effect of training programs. Rev Econ Stat. 1985;67:648–60.

Donald SGLK. Inference with difference-in-differences and other panel data. Rev Econ Stat. 2007;89(2):221–33.

Imbens GWJ. Recent developments in the econometrics of program evaluation. J Econ Lit. 2009;47(1):5–86.

Goldfarb SS, Houser K, Wells BA, Brown Speights JS, Beitsch L, Rust G. Pockets of progress amidst persistent racial disparities in low birthweight rates. PLoS ONE. 2018;13(7):e0201658.

Prasannan L, Rochelson B, Shan W, Nicholson K, Solmonovich R, Kulkarni A et al. Social Determinants of Health and Coronavirus Disease 2019 in pregnancy: condensation: social determinants of health, including neighborhood characteristics such as household income and educational attainment, are associated with SARS-CoV-2 infection and severity of COVID-19 in pregnancy. Am J Obstet Gynecol MFM. 2021:100349.

Flores LE, Frontera WR, Andrasik MP, Del Rio C, Mondriguez-Gonzalez A, Price SA, et al. Assessment of the Inclusion of Racial/Ethnic Minority, female, and older individuals in Vaccine clinical trials. JAMA Netw Open. 2021;4(2):e2037640.

Janevic T, Glazer KB, Vieira L, Weber E, Stone J, Stern T, et al. Racial/Ethnic disparities in very Preterm Birth and Preterm Birth before and during the COVID-19 pandemic. JAMA Netw Open. 2021;4(3):e211816.

Krans EE, Kim JY, James AE 3rd, Kelley D, Jarlenski MP. Medication-assisted treatment use among pregnant women with opioid Use Disorder. Obstet Gynecol. 2019;133(5):943–51.

Parekh NJM, Kelley D. Disparities in access to care and emergency department utilization in a large Medicaid program. J Health Dispar Res Pract. 2018;11(4):1–13.

Assibey-Mensah V, Fabio A, Mendez DD, Lee PC, Roberts JM, Catov JM. Neighbourhood assets and early pregnancy cardiometabolic risk factors. Paediatr Perinat Epidemiol. 2019;33(1):79–87.

Mendez DD, Hogan VK, Culhane JF. Institutional racism, neighborhood factors, stress, and preterm birth. Ethn Health. 2014;19(5):479–99.

Mendez DD, Hogan VK, Culhane J. Institutional racism and pregnancy health: using Home Mortgage Disclosure act data to develop an index for mortgage discrimination at the community level. Public Health Rep. 2011;126(Suppl 3):102–14.

Huffstetler AN, Phillips RL. Jr. Payment structures that support Social Care Integration with Clinical Care: Social Deprivation indices and Novel Payment models. Am J Prev Med. 2019;57(6 Suppl 1):S82–8.

Anderson AC, O’Rourke E, Chin MH, Ponce NA, Bernheim SM, Burstin H. Promoting Health Equity and eliminating disparities through performance measurement and payment. Health Aff (Millwood). 2018;37(3):371–7.

Jarlenski M, Borrero S, La Charite T, Zite NB. Episode-based payment for Perinatal Care in Medicaid: implications for practice and policy. Obstet Gynecol. 2016;127(6):1080–4.

McConnell KJ, Charlesworth CJ, Meath THA, George RM, Kim H. Oregon’s emphasis on Equity Shows Signs of Early Success for Black and American Indian Medicaid Enrollees. Health Aff (Millwood). 2018;37(3):386–93.

Hood CM, Gennuso KP, Swain GR, Catlin BB. County Health rankings: relationships between determinant factors and Health outcomes. Am J Prev Med. 2016;50(2):129–35.

Download references

This study received funding from the National Institute of Nursing Research under award R01NR020670. The funder had no role in the study design, data collection or analysis, or decision to publish the study.

Author information

Authors and affiliations.

Department of Health Policy and Management, University of Pittsburgh School of Public Health, 130 DeSoto St, A619, 15261, Pittsburgh, PA, USA

Marian Jarlenski, Evan Cole & Christine McClure

Department of Epidemiology, University of Pittsburgh School of Public Health, Pittsburgh, PA, USA

Sarah Sanders, Marquita Smalls & Dara D Méndez

You can also search for this author in PubMed   Google Scholar

Contributions

Jarlenski: Conceptualization; funding acquisition; investigation; methodology; supervision; writing-original draftCole: Conceptualization; data curation; investigation; resources; writing-reviewing and editingMcClure: Investigation; project administration; supervision; writing-reviewing and editingSanders: Investigation; methodology; visualization; writing-reviewing and editingSmalls: Investigation; project administration; visualization; writing-reviewing and editingMendez: Conceptualization; funding acquisition; investigation; validation; supervision; writing-original draft.

Corresponding author

Correspondence to Marian Jarlenski .

Ethics declarations

Ethical approval, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Jarlenski, M., Cole, E., McClure, C. et al. Implementation and early effects of medicaid policy interventions to promote racial equity in pregnancy and early childhood outcomes in Pennsylvania: protocol for a mixed methods study. BMC Health Serv Res 24 , 498 (2024). https://doi.org/10.1186/s12913-024-10982-5

Download citation

Received : 12 December 2023

Accepted : 10 April 2024

Published : 22 April 2024

DOI : https://doi.org/10.1186/s12913-024-10982-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Well-child visits
• Prenatal care
• Health policy
• Health equity
• Mixed methods

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Open access
• Published: 29 April 2024

Development and psychometric evaluation of "Caring Ability of Mother with Preterm Infant Scale" (CAMPIS): a sequential exploratory mixed-method study

• Saleheh Tajalli   ORCID: orcid.org/0000-0002-2045-6430 1 ,
• Abbas Ebadi   ORCID: orcid.org/0000-0002-2911-7005 2 ,
• Soroor Parvizy   ORCID: orcid.org/0000-0002-9361-9923 3 , 4 &
• Carole Kenner   ORCID: orcid.org/0000-0002-1573-5240 5  

BMC Nursing volume  23 , Article number:  297 ( 2024 ) Cite this article

Metrics details

Caring ability is one of the most important indicators regarding care outcomes. A valid and reliable scale for the evaluation of caring ability in mothers with preterm infants is lacking.

The present study was conducted with the aim of designing and psychometric evaluation of the tool for assessing caring ability in mothers with preterm infants.

A mixed-method exploratory design was conducted from 2021 to 2023. First the concept of caring ability of mothers with preterm infants was clarified using literature review and comparative content analysis, and a pool of items was created. Then, in the quantitative study, the psychometric properties of the scale were evaluated using validity and reliability tests. A maximum likelihood extraction with promax rotation was performed on 401 mothers with the mean age of 31.67 ± 6.14 years to assess the construct validity.

Initial caring ability of mother with preterm infant scale (CAMPIS) was developed with 64 items by findings of the literature review, comparative content analysis, and other related questionnaire items, on a 5-point Likert scale to be psychometrically evaluated. Face, content, and construct validity, as well as reliability, were measured to evaluate the psychometric properties of CAMPIS. So, the initial survey yielded 201 valid responses. The three components: 'cognitive ability'; knowledge and skills abilities'; and 'psychological ability'; explained 47.44% of the total observed variance for CAMPIS with 21 items. A subsequent survey garnered 200 valid responses. The confirmatory factor analysis results indicated: χ2/df = 1.972, comparative fit index (CFI) = 0.933, and incremental fit index (IFI) = 0.933. These results demonstrate good structural, convergent, discriminant validity and reliability. OMEGA, average inter-item correlation (AIC), intraclass correlation coefficients (ICC) for the entire scale were at 0.900, 0.27 and 0.91 respectively.

Based on the results of the psychometric evaluation of CAMPIS, it was found that the concept of caring ability in the Iranian mothers with preterm infants is a multi-dimensional concept, which mainly focuses on cognitive ability, technical ability, and psychological ability. The designed scale has acceptable validity and reliability characteristics that can be used in future studies to assess this concept in the mothers of preterm infants.

Peer Review reports

Introduction

In recent years, the technology advancements in the field of caring for mothers during pregnancy and delivery, and their infants resulted in increased survival of preterm infants [ 1 ]. According to the assessment of the world health organization (WHO), the prevalence of preterm birth globally is 10.6% [ 2 ].

Preterm birth is often an unexpected event for women [ 3 ], and hospitalization in the NICU is considered unavoidable [ 4 ]. The birth of a preterm infant and hospital stay in the NICU is very stressful for parents. The source of this stress includes the medical condition and parental separation from their infant. Mother-infant separation often results in the feelings of anxiety, fear, depression thus potentially decreasing the mother’s sensitivity to infant cues, which in turn, may delay infant development [ 5 ]. Many mothers experience acute stress disorder, which can last from 3 days to 1 month. It can be associated with symptoms such as troublesome memories, frequent and annoying restlessness and sadness, as well as post-traumatic stress [ 6 ].

The feeling of loss is a source of suffering and grief for the mothers of preterm infants. In the initial days following birth, the primary focus of parents is often on the infant's chances for survival. However, as time passes, this preoccupation can evolve into a potential cause of grief, anxiety, and feelings of culpability (related to an incomplete pregnancy)[ 6 ]. Therefore, in some cases, the mothers cannot focus on their preterm infants. Becoming a mother requires reorganizing and changing one’s personal identity. Mothers must change their identity from being a daughter and a wife in the generation to which they belong to being a mother for the next generation. They start this process from the pregnancy period, initially fantasizing about motherhood [ 7 ]. However, at the end of the pregnancy period, with the birth of a preterm infant, this process stops. In fact, preterm birth is a sudden stop in the process of mother's self-representation [ 8 ]. Consequently, the experience of transitioning into motherhood for women who give birth prematurely may differ from that of others. This is due to the fact that the psychological readiness and preparation required for assuming the role of a mother comes to an abrupt halt upon the premature birth of their infant. As a result, these women are forced to quickly adapt to their new circumstances [ 9 ].

Review of literature shows caring ability concept have three dimensions: cognitive, knowledge and patience [ 10 ]. Published caring ability scales evaluate ability of take care in informal caregiver of patient with cancer named ‘caring ability of family caregivers of patients with cancer scale (CAFCPCS)’ [ 11 ] or professional care giver named ‘The caring ability inventory’[ 10 ]. Also, published tools, focused on others aspects of take caring ability. So family empowerment tool named ‘Family Empowerment Scale’ [ 12 ], asses the empowerment of family members of patients with chronic disease as an outcome of taking care empowerment. The parent Engagement tool is another scale named ‘parent risk evaluation and engagement model and instrument (PREEMI)’ that evaluates parent risk evaluation and engagement of mothers with preterm infants [ 13 ]. The discharge preparedness scales were ‘perceived readiness for discharge after birth scale (PRDBS) and ‘Parent discharge readiness’ [ 14 ], that evaluate readiness for discharge. These evaluate postpartum mother’s perceptions of readiness for discharge from the hospital that was adapted from a scale measuring adult and elderly postsurgical patients’ perceptions of their readiness for discharge. Also assert for readiness is different from the ability to continue taking care after discharge. The scales 'Perceived maternal parenting self‐efficacy (PMP S‐E)' and 'Self‐efficacy in infant care scale' [ 15 , 16 ] were developed to assess the caregiver's belief in their ability to effectively care for their child, with a specific emphasis on self-confidence. These scales aim to provide a means of measuring the caregiver's expectations regarding the outcomes of their caregiving efforts. Although our previous study show a mother with optimal caring ability has sufficient cognitive ability, technical ability, and psychological ability [ 17 ]. This is a clear gap in the designed scale that could be overcome by a new scale that focuses take caring ability of mother with preterm infants. To sum up, there is no scale developed for the caring ability of mothers with preterm infants; therefore, the purpose of this study is to examine the real-life experiences of mothers with preterm infants and other professional and family. Caregivers, who have directly experienced taking care of preterm infants in order to then, design and perform the psychometric evaluation of a tool to assess the caring ability of mothers with preterm infants.

The purpose of this study was to develop a preterm caring ability scale and to examine its psychodynamic properties in mother of preterm infants with gestational age less than 32 weeks.

This study utilized a mixed-method exploratory design to develop and psychometric evaluate the caring ability of mother with preterm infant scale (CAMPIS) from July 2021 to October 2023. The research involved both mothers and professionals caring for preterm infants. The research consisted of two main phases: first, a qualitative study was conducted to generate the scale items, followed by a quantitative approach to evaluate the psychometric properties of the scale.

First step: qualitative study and item generation

The purpose of this step was to explain the concept of caring ability in the mothers of preterm infants, and to create a set of items to design the target scale. This step involved the identification of concepts through literature review published from 1995 to 2020 [ 17 ] .

In the second step, 18 semi-structured individual interviews of 60–80 min were organized to understand better the caring ability of mother with preterm infant concept. The face-to-face semi-structured interviews were conducted in a serene setting, either in the hospital room or the participants' home, as per their request and preference. The interviews involved mothers, grandmothers, and fathers, and were carried out without the presence of any other individuals. Due to the coronavirus social distance limitation, the interviews with the physicians and the nurses were conducted online and also recorded via Skyroom software.

We decided to finish the interview, when we interviewed the 18 participants, the information he/she provided is similar to those provided by the former ten participants (data saturation). Totally, 11 mothers, 2 fathers, 2 grandmothers of preterm infants, 1 neonatal nurse, and 2 neonatologists working in these wards participated in the present study. The participants were residents in the NICU in 5 hospitals in Tehran, Iran, representing a range of ages, genders, and caring role. The interview process started with a general question such as "Would you please explain mothers’ ability to take care of a preterm infant?", "In which situations do you feel she are more capable?", and "Which factors decreased her ability?" Then, based on the participant’s responses, the interviews continued with exploratory questions such as "Could you please explain more? or “Could you give an example in this regard?".

All participants were interviewed individually and each interview lasted between 60 and 80 min. The texts of the interviews were analyzed using the using Lindgern et al. [ 18 ] approach by MAXQDA software version 10. Qualitative interview resulted in initial items generation.

Second step: quantitative study and CAMPIS psychometric properties evaluation

Face validity.

Face validity was evaluated through qualitative and quantitative approaches. In the qualitative approach, the scale was sent to 10 mothers of preterm infants, they were asked to evaluate the scale in terms of difficulty, relevance, and ambiguity. The participants assessed the items based on their own judgment, ensuring that they were able to understand them. In order to further evaluate the suitability of the items, five additional mothers were included in the quantitative approach. These mothers were asked to rate the items on a 5-point Likert scale, ranging from completely suitable to not suitable at all. The impact score was calculated through the following equation: impact score = frequency (%) × appropriateness. A score above 1.5 was considered acceptable [ 19 ].

Content validity

The content validity of CAMPIS was evaluated through quantitative and qualitative approaches. In the qualitative approach, the scale was distributed among 22 neonatal nursing specialists, neonatal subspecialists, and scale development specialists to evaluate the items in terms of grammar and wording, item allocation, and scaling.

Then the content validity of the scale was modified by measuring content validity ratio (CVR) and content validity index (CVI) to ensure that the scale measures the intended construct in two separate stages. So, the 26 specialists, as highly knowledgeable about the mother of preterm infant or scale development, were asked to evaluate the items regarding necessity and relevancy. In CVR, 12 specialists evaluated the necessity of CAMPIS on a 3-point Likert scale (1 = not necessary, 2 = useful but not necessary, and 3 = necessary). CVR was calculated through the following formula: [ne – (N/2)]/(N/2), where “ne” is the number of the experts who rate the items as “essential”, and N is the total number of the items. The result was interpreted using Lawshe's content validity ratio [ 20 ].

Following the implementation of the required modifications based on the feedback provided by experts, the effectiveness of CAMPIS was further evaluated by 14 additional specialists in terms of the CVI. This evaluation was conducted using a four-point Likert scale, where a score of 1 indicated irrelevance, 2 denoted relative relevance, 3 represented relevance, and 4 signified complete relevance. The I-CVI, Kappa statistic, and S-CVI/Ave were computed to assess the content validity index at both the item-level and scale-level. A Kappa value exceeding 0.75 was deemed indicative of excellent agreement [ 19 ].

Item analysis

Before construct validity of the structure, the items were analyzed to identify the possible problems. At this step, 48 mothers of preterm infants, with the mean age of 31.88 ± 5.9 years, were selected through convenience sampling and enrolled. They were asked to identify that there were problems such as inappropriate reverse questions. Also they were asked to completed the hardcopy of CAMPIS and item-total correlations was evaluated for some items. A correlation coefficient lower than 0.32 or above 0.9 was considered as the criteria for removing the items [ 19 ].

Participants

The sample included the mothers of the preterm infants with a gestational age < 34 weeks. The criteria for entering the study consisted of the infant’s having been hospitalized in the NICU for more than two weeks, the infant’s not suffering from any major congenital anomalies, giving consent to participate in the study, and being able to use social networks such as WhatsApp. Based on the rule of thumb, that is, which considers 200 participants as an appropriate sample size [ 19 ], 401 mothers were considered for two phases at this step: 201 for the exploratory factor analysis (EFA) assessment, and 200 for the confirmatory factor analysis (CFA).

The participants were selected using convenience sampling through being in the hospital on the day of discharge, membership in the social groups related to following up mothers with preterm infants discharged from the NICU, and recommendations. In this step, data was collected online. For this purpose, an online questionnaire was created through the Porsline form, and its URL link ( https://survey.porsline.ir/s/d8uW4Jp ) was sent to the participants through the Telegram or WhatsApp social network applications (as the most common social networks among Iranian users).

The questionnaire used in this step included two parts. The first part was related to the infant's demographic characteristics, such as the infant's gender, and gestational age, and the mother's demographic characteristics, such as the mother's age, the infant's age, the mother's education, mothers' parity, assisted reproductive methods, the type of delivery, and previous experience in caring for infants. The second part included initial CAMPIS, with 38 items, for measuring the concept of the caring ability of mothers with preterm infants, on a five-point Likert scale (1 = never to 5 = always).

Construct validity

The construct validity of this scale was evaluated using EFA and CFA by SPSS 26 (SPSS Inc., Chicago, IL, USA). According to normal distribution of variables (skewness of ± 3, kurtosis of ± 7 and Mardia's coefficient less than 20) EFA was evaluated through Maximum likelihood factor analysis using Promax rotation. In addition, Kaiser–Meyer–Olkin (KMO) and Bartlett tests were used to estimate the adequacy and the appropriateness of the sample. KMO values higher than 0.9 were interpreted as excellent [ 19 ]. In order to extract the factors according to Thompson and Daniel recommendation [ 21 ] multivariate approach was used to identify the number of factors to extract in the EFA.

Then the factors were analyzed using the method of maximum likelihood analysis, which is one of the most common methods of data reduction. At first, 5 factors had an eigenvalue greater than 1, but considering that 3 factors explained an eigenvalue greater than 1.5 and a variance greater than 5%.

To extract the factor structure, exploratory graph analysis was used. The actual values ​​of the matrix were compared with the randomly generated matrix. The number of the components which have a higher variance in comparison with the components obtained from random data, after successive repetitions, is considered as the correct number of factors for extraction [ 22 ]. A factor loading of approximately 0.3 was considered to determine the presence of an item in a latent factor, and the items with communalities < 0.2 were excluded from EFA. The factor loading was estimated using the following formula: CV = 5.152 ÷ √ (n—2), where CV is the number of the extractable factors, and N is the sample size.

In the next step, the factor structure determined by EFA was evaluated by CFA. For this purpose, CFA was evaluated using Maximum likelihood factor analysis and the most common goodness-of-fit indices using SPSS/AMOS 26 software [ 22 ].

To discuss the fitness of the model on CFA, we can consider the various criteria for model fit indices. It has been suggested that Root Mean Square Error of Approximation (RMSEA) values less than 0.05 are good, and values between 0.05 and 0.08 are acceptable [ 23 ]. Therefore, the RMSEA value of 0.059 in this sample indicates an acceptable fit. The goodness of fit index (GFI) value of this sample, 0.88, is below 0.9, but the GFI is known depending on the sample size [ 24 ]. The frequency interference index (RFI) value, 0.085, is close to 0.9, which shows a relatively good fit [ 25 ]. The other fit indices, comparative fit index (CFI), incremental fit index (IFI), and Tucker-Lewis index (TLI), should be over 0.9 and parsimonious normed fit index (PNFI) should be over 0.5 for a good fit [ 25 ].

Reliability

The reliability was evaluated using internal consistency, stability, and absolute reliability approaches with SPSS 26. The internal consistency was evaluated using, McDonald's omega (Ω), and average inter-item correlation (AIC). The McDonald Omega of 0.7 or above, and the AIC of 0.2 to 0.4 were considered as acceptable criteria to evaluate the internal consistency. The consistency of CAMPIS was evaluated through calculating intraclass correlation coefficients (ICC) using a two-way random effects model [ 19 ]. The retest method with a time interval of 72 h (the day before discharge and 72 h after discharge) was used in 48 mothers with preterm infants. An ICC value > 0.8 is considered as the acceptable value for stability. In addition, the absolute reliability was evaluated using the standard error of measurement with the formula Standard Error of Measurement (SEM) = SD √ (1- ICC).

Finally, the responsiveness was evaluated using minimum detectable change (MDC) with the formula MDC = SEM × Z × √ 2, and minimal important change (MIC) with the following formula: MIC = SD × 0.5. If the MIC is smaller than the MDC, the scale is responsive. Besides, the interpretability was evaluated through calculating the MDC and testing the hypothesis [ 22 ].

Ethical consideration

The present study was extracted from the nursing Ph.D. thesis fund by Nursing Care Research Center (NCRC), School of Nursing and Midwifery, Iran University of Medical Sciences, Tehran, Iran. All ethical considerations of the study were approved by the ethics committee at the University of Medical Sciences (IR. IUMS. REC.1398.1407).

In qualitative step a total of 43 articles were selected in the study after reviewing 23291 extracted articles. We explored attribute of take caring ability concept. Findings showed a mother with optimal caring ability has sufficient knowledge, high skills, a sense of sufficient self-efficacy [ 17 ]. This step resulted in 69 initial items generation.

Using the findings of the literature review, comparative content analysis, and other related questionnaire items, the research teammates designed the initial items to measure the caring ability of mothers with preterm infants. An example of item generation has been presented in Table  1 . Then all the initial items (n: 104) were reviewed, and the generation of items was completed. In the selection of items, the focus was mainly on the features of the concept. The items of the pool were examined during joint meetings with the research team, and the items that were not in line with the purpose of this study were omitted according to experts’ opinions. First, repetitive descriptive items were deleted. For example, “I have access to healthcare providers and whenever I have questions, I can ask them” and “A healthcare provider answers my questions regarding the care of my child all day long 7 days a week.” This step resulted in 71 items. In the next step, the items with similar descriptions items were combined. For example, “I simply understand my child's needs.” and “When my child severely cries, I can figure out what my child wants.”

Therefore, at this step, CAMPIS was developed with 64 items, descriptions about behaviors and attitudes, on a 5-point Likert scale (always, most of the time, sometimes, rarely, never) to be psychometrically evaluated. Face, content, and construct validity, as well as reliability, were measured to evaluate the psychometric properties of CAMPIS.

In quantitative study and CAMPIS psychometric properties evaluation phase the impact score evaluation of face validity step, score of all items were above 1.5, and was considered appropriate. According to degree of mothers’ judgement all items of CAMPIS were appropriate to assessment of take caring ability. Therefore, no item was deleted.

During the process of content validity, some items were modified according to panelist feedbacks. While evaluating content validity, in the qualitative approach, 8 items were merged into one item based on the suggestion of the expert panel. Regarding number of participants for CVR were 12, the minimum acceptable CVR score was 0.56. In this step, the CVRs of 7 items were less than 0.56 and removed. In the CVI assessment, the I-CVI for all the items was in the range of 0.83–1, the modified Kappa was in the range of 0.84–1, and the total S-CVI/Ave and S-CVI/UA were 0.93 and 0.40 respectively. In content validity step according to the results, the Kappa values of 8 items were lower than 0.75, so they were removed. Therefore, 23 items were eliminated and the total number of CAMPIS was reduced from 64 to 41 items in content validity evaluation step.

In the item analysis step, the item-total correlation for 3 items was 0.32 or less, therefore were removed. The final CAMPIS with 38 items entered the factor analysis step.

In construct validity step, totally 401 mothers with a gestational age of 34 weeks or less participated in the present study. Their infants did not have any major surgical problems/anomalies, and had been hospitalized in the NICU for more than two weeks. The mean age of the mothers was 31.77 ± 6.02 years, with the minimum age of 16 and the maximum of 53 years. Out of 401 mothers, 204 (50.87%) had other children. All of them were married and lived with their spouses. The details of the socio-demographic characteristics of the participants have been shown in Table  2 .

In the construct validity phase, based on the results, the sample’s KMO and Bartlett's values were sufficient and appropriate, 0.897 and 1758.593, respectively ( P  ≤ 0.001). In this step, 17 items were removed as their shared values ​​were less than 0.2 and their factor loadings were less than 0.3. After Promax rotation, 3 factors (totally 21 items) were extracted: cognitive ability (9 items), technical ability (7 items), and psychological ability (5 items). These factors respectively explained 30.59, 9.62, and 7.28% of the total variance (47.44%) of the concept of the caring ability of mothers with preterm infants. The details of the factor analysis result have been presented in Table  3 .

Based on CFA indices, this sample has an acceptable fit to the 3 factors model and all of these indices in our study are excellent. The results of model fit indices have been given in Table  4 .

Prior to modeling modification, the goodness of fit measures for the CFA-generated 3-factor model indicated that the model fit but not optimally. To improve the factor structure model, we identified the following item content redundancies: Item 24 (If I see signs of an feeding difficulties, I know what to do) is related to Item 23 (I can recognize the signs and symptoms of shortness of breath and cyanosis), and item 8 (Given that my baby was born preterm, I am aware of the differences in growth and development with other babies.) is related to item 7 (I have enough information about my baby being preterm and its complications.).

Also, item 18 (I'm afraid my baby might be harmed.) is related to item 17 (I do not sleep well because I am worried about my baby's health.) Given the similarities of conceptual meaning, these correlated error terms indicated that these variables may share specific variances.

As shown in Fig.  1 , the aforementioned changes improved the goodness of fit of the model. This indicated the model of the caring ability of mothers with preterm infants fits the data (Fig.  1 ).

The results of CFA showed that a three-factor model of the care ability of the mothers with preterm infants indicated that the model fitted well. F1: Cognitive Ability; F2: Knowledge and Skills Abilities; F3: Psychological Ability

The retest method with a time interval of 72 h (the day before discharge and 72 h after discharge) was used in 48 mothers with preterm infants. An ICC value > 0.8 is considered as the acceptable value for stability. In addition, the absolute reliability was evaluated using the standard error of measurement with the formula Standard Error of Measurement (SEM) = SD √ (1- ICC). The details of the responsiveness are reported in Fig.  1 .

The results of McDonald Omega coefficient (ω = 0.90), and AIC (0.27) for the 3 factors were excellent. Based on the result, the ICC was 0.91 in the confidence interval of 0.84%-0.95%, which shows that the tool has acceptable measurement stability over time. Based on the SEM results, the absolute reliability was 3.18. This value shows that the scale scores of an individual vary ± 3.18 in repeated tests (Table  5 ). Based on the results, MDC = 8.78, MDC% = 11.79 and MIC = 0.50, this scale is responsive and interpretable.

The study’s results demonstrated that the concept of caring ability in mothers with preterm infants has three dimensions: cognitive ability, knowledge and skills abilities, and psychological ability. Therefore, CAMPIS is a valid and reliable scale to assess this concept in mothers with preterm infants. This scale includes 21 items, and three factors, cognitive, knowledge and skills, and psychological abilities, which explained 47.44% of the total variance of this concept. The CAMPIS model obtained through EFA was confirmed using CFA.

CAMPIS has three factors: "cognitive ability ", "knowledge and skills abilities" and "psychological ability". The first factor extracted from the scale was "cognitive ability". This factor includes 9 items regarding perception skills, cognitive skills, decision-making skills, movement skills, and attitude, which were extracted with the highest variance (30.59%). Cognitive abilities are the skills that a person needs to do anything from the simplest to the most complex, including perception skills, decision-making skills, movement skills, language skills, and social skills [ 26 ]. Cognitive abilities are the link between behavior and the brain structure, which include a wide range of abilities including planning, paying attention, problem solving, performing tasks simultaneously, and cognitive flexibility [ 27 ]. In this scale, cognitive ability was defined as the ability of a mother with a preterm baby to use skills which help her process information, think, reason, and solve problems faster and more efficiently. By developing cognitive skills, a person can go through the process of reasoning, decision making, and taking action. In this way, she can make sure that she can perceive the new situation, and perform her role effectively. The findings of a study conducted in 2004 showed that the mother’s possessing the desired attitudinal-cognitive ability had a significant impact on the infant’s health [ 28 ].

The second factor extracted from the scale was "technical ability". This factor includes 7 items regarding care knowledge and skills, as well as the ability to apply them; it was extracted with an acceptable variance (9.62%). In the science of care, knowledge is defined as the caregiver's awareness of the care recipient’s needs, strengths, and weaknesses as a unique member [ 10 ]. Moreover, in a study conducted by Galvin et al. in 2017, which was conducted with the aim of investigating the analytical features of the concept of readiness for discharge from the hospital, having sufficient knowledge and skills was mentioned as one of the characteristics of readiness for discharge [ 29 ]. In this scale, knowledge-skill ability is defined as that the mother’s awareness of the tasks that she must know how to perform in order to provide effective, efficient, and reliable care for her infant. The mothers of preterm infants often have poor knowledge regarding infant care after discharge [ 30 ], while the main factor which shows the mother's ability to provide care for the infant is having sufficient knowledge and skills. The mother should know what, how, why, and when to provide care for her infant [ 30 ].

The lived experience of the parents with preterm infants showed that in most cases, they are not prepared for the infant’s birth. The birth of a preterm infant puts them in a special situation which requires new care skills [ 31 ]. The parents need to acquire these skills that are a prelude to the discharge and transfer of the infant to home [ 32 ]. Providing quality care for complex disorders requires specialized caregiving knowledge and skills. When there is a deficiency in these areas, negative psychological impact on the parents and their relationship with their infant can ensue [ 33 ]. Studies have shown that having sufficient preparation for discharge and transitional care can help the families of preterm infants with a successful transition from the hospital to the family, reducing the rehospitalization rates [ 34 ]. To this end, health care professionals should provide the parents with useful information regarding illness management, strengthen their relationships with the hospital staff, encourage sharing experiences and emotions, and perform home visits [ 35 ]. The empowerment method should be appropriate to the caregiver's conditions. It should predict the caregiver's psychological condition as well as his/her emotional and behavioral responses. It should be used to train and support the caregiver in decision-making and managing the caregiving situation, considering his/her experiences, social status, cultural level, and beliefs [ 36 ].

The third factor extracted from the scale was "psychological ability". This factor includes 5 items regarding the psychological characteristics of the mothers taking care of preterm infants, with an acceptable variance (7.28%). An individual who is aware of his/her abilities and need for self-dependence, to manage care after discharge, has good psychological ability [ 29 ]. It is necessary to achieve psychological ability for an individual to deal with post-discharge challenges and have control over the situation [ 37 ]. In the present study, the mother's psychological ability is defined as her possessing the psychological characteristics and mental makeup to feel ready to continue care provision after discharge from the NICU, and to be able to fulfill her caring role successfully. During a traumatic event such as an infant's hospitalization in the NICU, the mothers try to improve their psychological ability [ 38 ].

The most mothers of preterm infants are not psychologically prepared for delivery and motherhood [ 39 ]. Preterm birth is considered a sudden and unpredictable event, which is accompanied by a feeling of shock and helplessness. The mothers of preterm infants often describe these conditions using terms such as falling to the bottom of a deep well, and being stuck in a whirlwind and storm happening around them; they admit they do not have enough control over what is happening, and lack the ability to take care of their infants [ 40 ]. Hospitalization in the NICU and mother-infant separation cause a feeling of inadequacy in the mother [ 41 ], which can subsequently affect her psychological ability. This maybe cause using different methods to deal with it. Using ineffective coping methods regarding changes in lifestyle and playing one’s role impacts on mother's caretaking tasks. If the use of ineffective coping methods is not recognized at the right time, and if appropriate measures are not taken, then the nursing diagnosis of coping disability disorder will be imminent [ 42 ]. Improved mental health and sufficient psychological ability are an important prerequisite for behavior change, which acts as a link between awareness and action. It can have a moderating role in empowering individuals, leading to positive thoughts, greater self-esteem and goals, more positive emotions and desirable behaviors [ 43 ]. Therefore, it is very important to measure the psychological ability of the mother in order for her to acquire the ability to provide quality care for the preterm infant.

Being aware of the caring ability of their mothers, as the main care givers, and designing an intervention to improve their caring ability can prevent negative side effects and help to improve the quality of care. The present study was conducted with the aim of designing and psychometric evaluation of the tool for assessing caring ability in mothers with preterm infants. Since one of the main goals of psychometric evaluation and factor analysis is to maximizing the explained variance by the model, in this research, the variance was 47.44%. Among the scales designed to measure caring ability, regardless of factor analysis extraction method, only one scale, the caring ability scale of the caregivers of cancer patients (67.7%), explains variance more than CAMPIS does [ 11 ].

In addition, this CAMPIS had very good internal consistency based on the results of Cronbach's alpha, AIC, and McDonald omega. It should be noted that one of the advantages of this scale is having strong stability based on the ICC value. Another advantage of this study was the assessment of measurement error, responsiveness, and interpretation of the CAMPIS. The results showed that the CAMPIS has the minimum amount of SEM, responsiveness, and interpretability. SEM shows the accuracy of the measurement for each individual, and it is important that this value be small. Responsiveness refers to the ability of a scale to reflect changes in an individual's position over a period. Finally, interpretability refers to the scale's ability to show the significance of changes. These characteristics are an important and necessary part of consensus-based standards for the selection of health measurement tools, which have not been reported in the previous studies on the psychometric characteristics of caring ability.

CAMPIS measures the caring ability of the mother of a preterm infant in the three factors namely ‘Cognitive Ability’ (items: 1–9), ‘knowledge and skills abilities' (items: 10–16), and ‘Psychological Ability’ (items: 17–21). The answer to the items is based on a five-point Likert scale (always (5), most of the time (4), sometimes (3), rarely (2), never (1)). Scoring of items 17,18,19,20 and 21 is reverse so (always (1), most of the time (2), sometimes (3), rarely (4), never (5)). To have an overall score: Sum cognitive ability + sum knowledge and skills abilities + sum psychological ability = Total score. The best way is to calculate the average score for every scale, and compare the results with the average score. In our studies we set the average score for cognitive ability from 9–45, knowledge and skills abilities 7–35, psychological ability 5–25, and overall score 21–105 (Additional file 1 : Appendix A). CAMPIS, is a useful scale for professional caregivers and researchers, thanks to its brief items, good variance, reliability, as well as exclusively belonging to this group.

The findings of this study demonstrated that CAMPIS is a reliable and valid scale with 21 items, which includes the 3 dimensions of attitudinal-cognitive ability, knowledge-skill ability, and psychological ability for measuring the concept of tenacity in family caregivers. Although the exclusiveness of CAMPIS to evaluate the caring ability of mothers with preterm infants is one of the points of strengths in this study, but considering that the samples were selected from the population of Iranian mothers with preterm infants with a gestational age of 34 weeks or less. It should be used with caution in mothers with preterm infants with a gestational age over 34 weeks. Therefore, one of the important limitations was the concern about the generalizability of the findings.

Limitations and strength

In the construct validity step, data collection was done online. Although the use of online questionnaires, especially during the period of social restrictions due to COVID-19 pandemic, has many advantages, such as the possibility of eliminating the missing data, speeding up the data collection process, and the possibility of collecting data from other provinces and counties. There are also some limitations like self-selection bias, and the lack of interaction with the participants. Furthermore, some qualified mothers were excluded from the study due to illiteracy, the lack of internet access, and the inability to use the phone/laptop to access social networks.

During the item generation step, we considered the different directions of items, but during data reduction, opposite directions were removed. Finally, all items of each factor of CAMPIS have oriented in the same direction, and it is imaginable to create a possibility for response tendency.

One of the problems of self-report scales is that they are subject to the respondent's interpretation of the items, which may not be what the scale designer has intended. In order to reduce this potential problem, continuous testing and modification of the scale has been done. Since ability is a personal matter, the participants were asked not to reveal their names, cities, and the name of the hospitals where their infants were hospitalized. This study has points of strength. One of them is the evaluation of SEM, ICC, responsiveness, and interpretability as important and required items of the COSMIN checklist, which had not been reported previously regarding caring ability scales, but were evaluated in this study.

Availability of data and materials

The original contributions presented in the study are included in the article. Request access to other supplementary material can be directed to the first or corresponding author.

Abbreviations

Caring Ability of Mother with Preterm Infant Scale

Exploratory factor analysis

Confirmatory factor analysis

Neonatal Intensive Care Unit

World Health Organization

Content validity ratio

Content validity index

Item-level content validity index,

Scale-level content validity index average

Kaiser–Meyer–Olkin

Average Inter-Item Correlation

Intraclass correlation coefficients

Minimum Detectable Change Analysis

Minimal Important Change

Comparative Fit Index

Incremental Fit Index

Frequency Interference Index

Parsimonious Normed Fit Index

Tucker-Lewis Index

Goodness of Fit Index

Root Mean Square Error of Approximation

Confidence Interval

Standard Error of Measurement

Cao Y, Jiang S, Sun J, Hei M, Wang L, Zhang H, et al. Assessment of neonatal intensive care unit practices, morbidity, and mortality among very preterm infants in China. JAMA Network Open. 2021;4(8):e2118904-e.

Article   Google Scholar  

Chawanpaiboon S, Vogel JP, Moller A-B, Lumbiganon P, Petzold M, Hogan D, et al. Global, regional, and national estimates of levels of preterm birth in 2014: a systematic review and modelling analysis. Lancet Glob Health. 2019;7(1):e37–46.

Article   PubMed   Google Scholar  

Lasiuk GC, Comeau T, Newburn-Cook C. Unexpected: an interpretive description of parental traumas’ associated with preterm birth. BMC Pregnancy Childbirth. 2013;13(1):1–10.

Google Scholar  

Howson C, Merialdi M, Lawn J, Requejo J, Say L. March of dimes white paper on preterm birth: the global and regional toll. March of dimes foundation. 2009:13.

Gerstein ED, Njoroge WF, Paul RA, Smyser CD, Rogers CE. Maternal depression and stress in the neonatal intensive care unit: associations with mother− child interactions at age 5 years. J Am Acad Child Adolesc Psychiatry. 2019;58(3):350-8. e2.

Article   PubMed   PubMed Central   Google Scholar  

Fowler N, Vo PT, Sisk CL, Klump KL. Stress as a potential moderator of ovarian hormone influences on binge eating in women. F1000Research. 2019;8.

Stern DN, Bruschweiler-Stern N. The birth of a mother: How the motherhood experience changes you forever: Basic Books; 1998.

Slade A, Cohen LJ, Sadler LS, Miller M. The psychology and psychopathology of pregnancy. Handbook of Infant Mental Health. 2009;3:22–39.

Spinelli M, Frigerio A, Montali L, Fasolo M, Spada MS, Mangili G. ‘I still have difficulties feeling like a mother’: the transition to motherhood of preterm infants mothers. Psychol Health. 2016;31(2):184–204.

Nkongho NO. The caring ability inventory. Measure Nurs Outcomes. 2003;3:184–98.

Nemati S, Rassouli M, Ilkhani M, Baghestani AR, Nemati M. Development and validation of ‘caring ability of family caregivers of patients with cancer scale (CAFCPCS).’ Scand J Caring Sci. 2020;34(4):899–908.

Koren PE, DeChillo N, Friesen BJ. Measuring empowerment in families whose children have emotional disabilities: a brief questionnaire. Rehabil Psychol. 1992;37(4):305.

Samra HA, McGrath JM, Fischer S, Schumacher B, Dutcher J, Hansen J. The NICU parent risk evaluation and engagement model and instrument (PREEMI) for neonates in intensive care units. J Obstet Gynecol Neonatal Nurs. 2015;44(1):114–26.

Smith V, Young S, Pursley D, McCormick M, Zupancic J. Are families prepared for discharge from the NICU? J Perinatol. 2009;29(9):623–9.

Article   CAS   PubMed   Google Scholar  

Barnes CR, Adamson-Macedo EN. Perceived maternal parenting self-efficacy (PMP S-E) tool: development and validation with mothers of hospitalized preterm neonates. J Adv Nurs. 2007;60(5):550–60.

Prasopkittikun T, Tilokskulchai F, Sinsuksai N, Sitthimongkol Y. Self-efficacy in infant care scale: development and psychometric testing. Nurs Health Sci. 2006;8(1):44–50.

Tajalli S, Ebadi A, Parvizy S, Kenner C. Maternal caring ability with the preterm infant: a Rogerian concept analysis. Nurs Forum. 2022;57(5):920–31.

Lindgren B-M, Lundman B, Graneheim UH. Abstraction and interpretation during the qualitative content analysis process. Int J Nurs Stud. 2020;108:103632.

Sharif Nia H, Zareiyan A, Ebadi A. Test Development Process in Health Sciences; Designing and Psychometric properties. Tehran: Jame-e-negar; 2022.

Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563–75.

Thompson B, Daniel LG. Factor analytic evidence for the construct validity of scores: A historical overview and some guidelines. Thousand Oaks: Sage Publications Sage CA; 1996. p. 197–208.

Pahlevan Sharif S, Sharif NH. Factor analysis and structural equation modeling with SPSS and AMOS. Tehran: Jame-e-Negar; 2020.

Fabrigar L, Wegener Dt, MacCullum R. Strahan, e. J. Evaluating the use of factor analysis in psychological research. Psychological Methods. 1999;4:272-99.

Mulaik SA, James LR, Van Alstine J, Bennett N, Lind S, Stilwell CD. Evaluation of goodness-of-fit indices for structural equation models. Psychol Bull. 1989;105(3):430.

Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238.

Benjamin DJ, Brown SA, Shapiro JM. Who is ‘behavioral’? Cognitive ability and anomalous preferences. J Eur Econ Assoc. 2013;11(6):1231–55.

Ones DS, Dilchert S, Viswesvaran C, Salgado JF. Cognitive abilities: Taylor & Francis Group; 2010. 255–75 p.

Rubalcava LN, Teruel GM. The role of maternal cognitive ability on child health. Econ Hum Biol. 2004;2(3):439–55.

Galvin EC, Wills T, Coffey A. Readiness for hospital discharge: a concept analysis. J Adv Nurs. 2017;73(11):2547–57.

Hochreiter D, Kuruvilla D, Grossman M, Silberg J, Rodriguez A, Lary L, et al. Improving guidance and maternal knowledge retention after well-newborn unit discharge. Hosp Pediatr. 2022;12(2):148–56.

Amorim M, Alves E, Kelly-Irving M, Silva S. Needs of parents of very preterm infants in Neonatal Intensive Care Units: a mixed methods study. Intensive Crit Care Nurs. 2019;54:88–95.

Jing L, Bethancourt C-N, McDonagh T. Assessing infant and maternal readiness for newborn discharge. Curr Opin Pediatr. 2017;29(5):598–605.

Hellesø R, Eines J, Fagermoen MS. The significance of informal caregivers in information management from the perspective of heart failure patients. J Clin Nurs. 2012;21(3–4):495–503.

Boykova M, Kenner C. Transition from hospital to home for parents of preterm infants. J Perinat Neonatal Nurs. 2012;26(1):81–7.

Lundqvist P, Weis J, Sivberg B. Parents’ journey caring for a preterm infant until discharge from hospital-based neonatal home care—A challenging process to cope with. J Clin Nurs. 2019;28(15–16):2966–78.

Hamilton DL. Personality attributes associated with extreme response style. Psychol Bull. 1968;69(3):192.

Carroll Á, Dowling M. Discharge planning: communication, education and patient participation. Br J Nurs. 2007;16(14):882–6.

Aftyka A, Rozalska-Walaszek I, Rosa W, Rybojad B, Karakuła-Juchnowicz H. Post-traumatic growth in parents after infants’ neonatal intensive care unit hospitalisation. J Clin Nurs. 2017;26(5–6):727–34.

Rossman B, Greene MM, Meier PP. The role of peer support in the development of maternal identity for “NICU moms”. J Obstet Gynecol Neonatal Nurs. 2015;44(1):3–16.

O’Donovan A, Nixon E. “Weathering the storm:” Mothers’ and fathers’ experiences of parenting a preterm infant. Infant Ment Health J. 2019;40(4):573–87.

Kestler-Peleg M, Lavenda O, Stenger V, Bendett H, Alhalel-Lederman S, Maayan-Metzger A, et al. Maternal self-efficacy mediates the association between spousal support and stress among mothers of NICU hospitalized preterm babies. Early Human Dev. 2020;146:105077.

Ackley BJ, Ladwig GB, Makic MBF, Martinez-Kratz M, Zanotti M. Nursing Diagnosis Handbook, Revised Reprint with 2021–2023 NANDA-I® Updates-E-Book: Elsevier Health Sciences; 2021.

Karademas EC. Self-efficacy, social support and well-being: The mediating role of optimism. Personality Individ Differ. 2006;40(6):1281–90.

Download references

Acknowledgements

This article extracted from Ph.D. thesis of the first author, which was financially supported by the Nursing and Midwifery Care Research Center in ……… University of Medical Sciences (NCRC-1407). The authors would like to extend their sincere thanks to mothers, healthcare providers and other participants who shared their experiences with us.

Clinical trial registration

Not applicable.

Preprint disclosure

Saleheh Tajalli, Abbas Ebadi and Carole Kenner have no funding to disclose. Soroor Parvizy is supported by a grant from the vice chancellor of research from Nursing and Midwifery Care Research Center in Iran University of Medical Sciences under Grant number [1398–12-27–1407] CRediT.

Author information

Authors and affiliations.

School of Nursing and Midwifery, Tehran University of Medical Sciences, Tehran, Iran

Saleheh Tajalli

Behavioral Sciences Research Center, Life Style Institute, Nursing Faculty, Baqiyatallah University of Medical Sciences, Tehran, Iran

Abbas Ebadi

Nursing and Midwifery Care Research Center, Pediatric Nursing Department, School of Nursing and Midwifery, Iran University of Medical Sciences, Tehran, Iran

Soroor Parvizy

Center for Educational Research in Medical Sciences (CERMS), Department of Medical Education, School of Medicine, Iran University of Medical Sciences, Tehran, Iran

School of Nursing and Health Sciences, The College of New Jersey, Ewing, NJ, USA

Carole Kenner

You can also search for this author in PubMed   Google Scholar

Contributions

S.T: Conceptualization, Investigation, Methodology, Software, Writing—original draft, Writing—review & editing. S.P: Data curation, Formal analysis, Investigation, Methodology, Software, Writing—original draft, Writing—review & editing, Validation, Funding acquisition. A.E: Investigation, Methodology, Project administration, Supervision, Validation, Writing review & editing, Resources. C.K: Writing -review & editing, Resources. All authors contributed to the article and approved the submitted version.

Corresponding authors

Correspondence to Abbas Ebadi or Soroor Parvizy .

Ethics declarations

Ethics approval and consent to participate, consent to participate and publication.

All parents of neonate in the study were informed of the study objectives and signed a written informed consent form.

Consent for publication

Consent to publish was obtained from the study participants.

Competing interests

The authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix a..

The last version of "Caring Ability of Mother with Preterm Infant Scale" (CAMPIS).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Tajalli, S., Ebadi, A., Parvizy, S. et al. Development and psychometric evaluation of "Caring Ability of Mother with Preterm Infant Scale" (CAMPIS): a sequential exploratory mixed-method study. BMC Nurs 23 , 297 (2024). https://doi.org/10.1186/s12912-024-01960-7

Download citation

Received : 24 November 2023

Accepted : 22 April 2024

Published : 29 April 2024

DOI : https://doi.org/10.1186/s12912-024-01960-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Caring ability
• Maternal caregiving
• Preterm infant
• Psychometric

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

• Study Protocol
• Open access
• Published: 30 April 2024

Swedish massage as an adjunct approach to Help suppOrt individuals Pregnant after Experiencing a prior Stillbirth (HOPES): a convergent parallel mixed-methods single-arm feasibility trial protocol

• Sarah Fogarty   ORCID: orcid.org/0000-0002-6358-5873 1 , 2 , 4 ,
• Alexander E. P. Heazell 2 , 3 ,
• Niki Munk 5 , 6 &
• Phillipa Hay 2 , 7  

Pilot and Feasibility Studies volume  10 , Article number:  67 ( 2024 ) Cite this article

Metrics details

Women experiencing pregnancy after stillbirth experience high levels of anxiety, fear and depression. Standard antenatal care may be emotionally unsuitable for many women at this time, and there is a lack of evidence on what interventions or approaches to care might benefit these women. Therapeutic massage may assist women after stillbirth by decreasing anxiety, worry and stress.

This paper outlines the objectives, methodology, outcome and assessment measures for the Helping suppOrt individuals Pregnant after Experiencing a Stillbirth (HOPES) feasibility trial which evaluates massage as an adjunct approach to care for pregnant women who have experienced a prior stillbirth. It also outlines data collection timing and considerations for analysing the data.

HOPES will use a convergent parallel mixed-methods, single-arm repeated measures trial design in trained massage therapists’ private clinics across Australia. HOPES aims to recruit 75 individuals pregnant after a previous stillbirth. The intervention is massage therapy treatments, and participants will receive up to five massages within a 4-month period at intervals of their choosing. Primary quantitative outcomes are the feasibility and acceptability of the massage intervention. Secondary outcomes include determining the optimal timing of massage therapy delivery and the collection of measures for anxiety, worry, stress and self-management. A thematic analysis of women’s experiences undertaking the intervention will also be conducted. A narrative and joint display approach to integrate mixed-methods data is planned.

The HOPES study will determine the feasibility and preliminary evidence for massage therapy as an intervention to support women who are pregnant after a stillbirth.

Trial registration.

ClinicalTrials.gov NCT05636553. Registered on December 3, 2022, and the trial is ongoing.

Peer Review reports

There are almost 6 stillbirths a day in Australia (1, 2) where stillbirth is defined as ‘in-utero death from a 20-week gestation until immediately before birth’ (3). Stillbirth profoundly alters the reality of subsequent pregnancies (4). Pregnancy after stillbirth has been described by mothers as stress that is ‘survived’. (5) The impact of stillbirth on subsequent pregnancies is profound with mothers commonly describing conflicted emotions, high levels of anxiety and stress, fear, isolation, and a lack of trust in a good outcome (5–8). Women who have experienced a stillbirth in their previous pregnancy are at increased risk of adverse pregnancy outcomes in subsequent pregnancies such as pre-term birth and low birthweight (7, 9). The mental health of women experiencing pregnancy following stillbirth is similarly affected; with women experiencing post-traumatic stress disorder symptoms (10) and significantly more depression and anxiety than pregnant women who had not experienced a stillbirth (11, 12). Standard antenatal care is emotionally unsuitable for many women in pregnancies following a stillbirth due to their increased need of psychosocial support (13). Negative experiences of standard antenatal care include exacerbated stress and reawakened traumatic memories (5). There are few specific specialist antenatal services for women to access in a subsequent pregnancy after a stillbirth (5, 14). The international stillbirth research community has highlighted concerns about the use of traditional randomised control trial (RCT) methodologies especially when evaluating psychosocial support interventions in a pregnancy following stillbirth (15). Specific ethical concerns raised include the constraints of having a ‘treatment as usual/standard care’ intervention and withholding potentially beneficial care from those who need it (15). Methodological constraints include large sample size requirements, particularly in studies where all groups receive the active intervention (16, 17) and ‘RCT methodologies being incompatible with a much-needed individualised approach to care’ (15). Evaluation concerns in this population have contributed to the lack of direct evidence on what specific interventions or approaches to care might benefit women experiencing pregnancy after stillbirth including adjunct interventions (15). More research is needed on interventions that benefit mothers and improve their emotional health during a pregnancy after stillbirth.

Pregnancy massage aims to support the physiologic, structural and emotional well-being of both mother and fetus using various massage techniques including the Swedish and remedial massage. There is growing evidence that massage can benefit emotional health in non-pregnant and pregnant populations in various ways including by decreasing anxiety (18–25), lowering stress (20, 22, 24, 26, 27), decreasing depression (19–23, 25, 28–30) and improving mood (22, 24, 25, 31, 32). Qualitative data for massage and bereavement found massage to be helpful for recipients by generating feelings of consolidation and cultivating the feeling of support and care (33, 34). There is limited research on the aggregate effects of massage care during therapeutic alliance building, one-on-one care over time and when individualised treatments are part of the ‘treatment’ alongside massage techniques or strokes (35). A recent case study found that a patient-centred massage treatment can be a support option for women experiencing a pregnancy after stillbirth with massage assisting the ability to cope during that stressful time (36).

Massage therapy is more than the application of a massage technique and is a philosophy of care including many non-manual factors; more research encompassing massage as a philosophy of care, particularly in the antepartum population, is needed. Massage has demonstrated a precursory capacity to address the somatic and psychological symptoms associated with being pregnant after a stillbirth as well as some of the altered psycho-behavioural factors that occur in a clinical context (36). Given the precursory evidence for massage as a support option for women pregnant after stillbirth and the highlighted ethical and methodological concerns regarding withholding potentially beneficial care, treatment as usual intervention groups, sample size requirements and the need for individualised treatments for this population, a study focused specifically on the feasibility of a massage therapy intervention trial in this population is warranted before embarking on larger and more expensive effectiveness research. This paper outlines the objectives, methodology, outcome and assessment measures for the Helping suppOrt individuals Pregnant after Experiencing a Stillbirth (HOPES) feasibility trial which evaluates individualised Swedish massage as an adjunct approach to care for pregnant women who have experienced a prior stillbirth.

The specific objectives of the study are as follows:

Assess the feasibility and acceptability of the massage intervention in the pregnant population who have a prior stillbirth experience;

Determine the optimal timing of therapy, and measurement collection needs for anxiety, worry, stress and self-management;

Conduct a thematic analysis of women’s experiences undertaking the intervention to evaluate the acceptability of the study processes and the intervention.

The HOPES feasibility trail began enrolment in February 2023 and expects to complete data collection in mid-2024.

The HOPES study utilizes a convergent parallel mixed-methods, single-arm repeated measures trial design aiming to assess the feasibility of Swedish massage as an adjunct approach to care for pregnant women who have previously experienced stillbirth. The convergent parallel mixed-methods design collects and analyses quantitative and qualitative individually, then compares and relates the two data types for areas of convergence or divergence and then interprets the meaning of the combined results. The primary quantitative outcomes are the feasibility and acceptability of the massage intervention. Secondary outcomes include determining the optimal timing of massage treatments and the collection of measures for anxiety, worry, stress and self-management. A thematic analysis of women’s experiences from a subsample of participants undertaking the intervention will also be conducted. The study intervention consists of up to five 60-min pregnancy massage treatments over a 4-month period at individualized time intervals per participant’s choice or availability. Each participant will complete outcome measures pre- and post-study intervention. Participants will complete a short outcome measure each pre- and post-massage treatment. A narrative and joint display approach to integrate mixed-methods data is planned.

Study design

The study is a convergent parallel mixed-methods, single-arm repeated measures pilot trial design.

Ethics approval and consent to participate

Human ethics approval for the study was granted by the Ethics Committee of Western Sydney University (No: H15261). Changes in protocol will undergo ethical approval. All study participants will receive written information about the study with the option for information to be provided orally. All study participants will give informed written consent for the use of their data and the application of the intervention. Participation in the study is voluntary and can be refused.

Data are treated confidentially and processed pseudonymously. The collection and storage of personal data take place in accordance with the University’s Research Data Management Policy. Shared information collected from the participants is non-identifiable using a study identifier (number). Quantitative data collected will be available for use in any other research projects in the future. To make reuse of the quantitative data possible, it will be stored under Western Sydney University’s Open Access Policy.

Recruitment and screening

Potential participants are recruited via social media and from maternity clinics, shared care General Practitioners and obstetrician clinics around the massage therapists’ locations within Australia. Maternity services and clinics approached are based on the proximity of services to the locations of the massage therapists participating in the study. Potential participants are informed about the study via their obstetric healthcare provider or via social media platforms for stillbirth support services such as SANDS (37) and Red Nose (38). Potential participants contact the researchers via the shared contact information who explain the study and screen potential participants for eligibility (see 2.4.1 and Table  1 ). Potential participants are given a clear explanation of the requirements of the study, the opportunity to ask questions and time to consider their participation. Individuals who agree to take part must sign an informed consent form. Consenting individuals meeting the criteria of the study are then enrolled in the study.

Eligibility criteria

A purposeful sample of pregnant women are being recruited for the HOPES study who have experienced a stillbirth or termination for medical reasons (TFMR) after 20 weeks of gestation in a previous pregnancy (see Table  1 ). To be eligible to participate, pregnant women need to be less than 30 weeks of gestation. There are no criteria for eligibility related to the interpregnancy interval between the stillbirth and the current pregnancy because evidence shows the impact of stillbirth can be felt for years, and sometimes decades, after the loss (8, 39–41). The mental health of participants experiencing pregnancy after a stillbirth is varied, and we chose not to limit access to the study based on only one or two of the many mental health symptom participants may experience (e.g., anxiety, stress); especially as the mental health of participants is expected to be impacted at different time points during the pregnancy and be unique to each individual and situation (39).

Sample size

The study aims to recruit a total of 75 pregnant people meeting the study criteria. This feasibility study is not expecting large effect sizes as anxiety and depression experienced during a pregnancy after stillbirth do not decrease until 6 months post birth (42). Thus, it is hypothesized that while the immediate effects of the massage intervention might be larger, the long-term effects will likely be small (< 0.1). Using the approach by Whitehead et al. for small effect sizes (43), it is estimated a minimum of 50 pregnant people is needed to reliably detect a small effect size (< 0.1). A sample of 75 pregnant women allows for a 10–15% drop out rate and 10% for whom data is incomplete.

A sub-group sample of 20 participants will be invited via purposive sampling to participate in the qualitative phase of the mixed-methods design. Twenty participants have been proposed as optimal to ‘improve open and frank exchange of information and mitigate some of the bias and validity concerns inherent in qualitative research’ (44). Purposive sampling will consider the type of loss (stillbirth and TFMR), early stillbirth/TMFR loss (losses between 20 and 29 + 6 weeks of gestation) and late stillbirth/TMFR loss (losses between 30 and birth).

Intervention

Each participant will have access to up to 5 allocated 60-min massage consultations see Fig.  1 . The massage treatments will be administered within a 4-month period at intervals of participant and therapist dyad determining based on participant preference and therapist treatment planning. Each massage consultation will be individualised to meet the needs of the participant on the day of treatment. The conceptual approach utilized is based on a vulnerability-to-stress concept. The vulnerability-to-stress concept acknowledges the impact of stress on vulnerability and recognises protective factors that can help reduce stress (45). For women pregnant after a stillbirth, the vulnerability is the previous loss (and any other pregnancy losses) as well as any biological factors, and the stress is being pregnant again (and the fear of another loss). We are proposing that massage might be a protective factor to help manage stress. This conceptual approach was used in a published case study (36). The massage aims to support women by addressing altered psycho-behavioural and physiological factors associated with vulnerability and stress such as countering feelings of loss of control via allowing participants to individualise their treatments (e.g. depth, area of the body to be treated, what they want to be treated). See Additional file 1 for the protocol for the massage which covers more in-depth how massage may do this. The massage intervention uses hands-on techniques, listening and creating the environment to build a therapeutic alliance to help provide protective factors and manage the stress of pregnancy after a stillbirth. Allowable massage strokes for therapist incorporation into treatment to address the study aims include longitudinal gliding, transverse gliding, digital ischemic pressure, transverse frictions and transverse gliding. The use of these massage strokes is based on prior pregnancy massage research (39, 42). The areas of the body treated, the depth of treatment and which massage strokes used will be determined by the massage therapist reflecting real-world clinical massage practice. No other form of massage therapy or hands-on modalities (e.g. reflexology, aromatherapy massage) are to be used and nor are other healing modalities (e.g. reiki). Treatment sessions are 60 min and include hands-on treatment time as well as the time prior and after the hands-on massage to consult with the participant about their treatment and plan the timing for the next treatment session. See Additional file 1.

CONSORT diagram of participant flow through the study

Delivery of the intervention

The massages will be provided by a team of massage therapists with experience in pregnancy massage and perinatal loss across multiple sites within Australia. Massage therapists will be recruited (1) through the study personnel who have previously worked with the therapists and or (2) through an expression of interest via a pregnancy massage training organisation in Australia. Therapists from all states and territories across Australia who had completed pregnancy massage training post-completion of their initial massage qualifications are able to express an interest in being study massage therapists. At least 20 massage therapists will be recruited to ensure that there are therapists in differing states and territories across Australia due to the difficulty of recruiting 75 people from a more limited geographic area. The exact number of therapists to be recruited is unknown as it is undetermined how many therapists will express an interest. The massage therapists attend an online 3-h training on the massage study intervention and facilitation of the massage experience for individuals pregnant after a loss as well as training on grief and resources available to individuals pregnant after a loss prior to becoming study massage therapists. The massage treatments are administered in the massage clinician’s office and follow a protocol that allows for treatment to be tailored for each participant depending on the presentation on the day. See Additional file 1 for massage protocol. The focus of each treatment is determined by the participant via the Measure Yourself Concerns and Wellbeing (MYCaW) patient-reported outcome measure (PROM) (46), and the massage therapist administers remedial and or relaxation massage techniques.

Side effects

Participants will be sent a questionnaire 24 to 48 h after each massage treatment asking about experienced side-effects from the massage treatment such as post-massage soreness, headaches, tiredness or any other side effects (please specify). These side effects are based on research into side effects, massage and pregnancy (47). Adverse events are recorded and reported quarterly to the Research Ethics Committee, and serious adverse events are required to be reported immediately (Additional file 2).

Criteria for discontinuing the intervention

The intervention will cease if participants are advised by their obstetric healthcare providers (i.e. on bed rest), if they are no longer pregnant or if the study participant delivers their baby prior to completing the allocated study intervention.

Assessments

The measures used to provide data to meet the three objectives of the study are as follows:

Data will be collected by the study team throughout the trial on the number of inquiries, number of participants, drop-out rates and compliance with completing the validated outcome measures. See Table  2 .

Treatment utilisation

This data will describe when study participants utilised their massages (massage timing) to inform hypothesis generation regarding dosing value.

Validated self-reported questionnaires

Worry, anxiety, coping, self-efficacy, stress and empathy will be assessed in the study via the following questionnaires:

Worry will be assessed via the Cambridge Worry Score (48) which is 17-item instrument that measures the content and extent of worries in pregnancy (48, 49). The total score range is from ‘zero to 85 with a higher score representing greater severity of worries’ (48).

Maternal anxiety symptoms will be assessed using the Generalized Anxiety Disorder Assessment (GAD-7) (50) which is a 7-item instrument that measures the severity of generalised anxiety disorder. The total score range is from zero to 21 with scores of 5–9 representing mild anxiety, 10–14 moderate anxiety and 15–21 representing severe anxiety (50) .

Coping via the self-reported Revised Prenatal Coping Inventory (NuPCI) (51) will be assessed. This scale has 32 items that evaluate the coping strategies of pregnant (51). The NuPCI consists of 3 subscales: positive attitude (15 items), avoidance (11 items) and spiritual-positive coping (6 items) (51). A higher score indicates more frequent use of a specific coping strategy (51).

Self-efficacy via the Strategies Used by People to Promote Health measure (SUPPH-29) (52) will be assessed. This scale has 29 items that measure self-care self-efficacy (52). The SUPPH-29 consists of 3 sub-scales positive attitude (16 items), stress reduction (10 items) and decision making (3 items) (52). The total score range is from 29 to 145 with a higher score indicating greater self-confidence to carry out self-care strategies (52).

Maternal stress symptoms via the Perceived Stress Scale (PSS) (53) will be assessed. The scale has 10 items that measure perceived personal stress (53). The total score range is from zero to 40 with a score ranging from 0 to 13 considered low stress, 14–26 moderate stress and 27–40 high stress (53).

Empathy in the context of the therapeutic relationship via the Consultation and Relational Empathy (CARE) patient-reported experience measure (PREM) (54). This scale has 10 items that measures patients’ perceptions of relational empathy in the consultation (54). The total score range is from 10 to 40 with higher scores indicating greater perceived relational empathy (54).

Measure Yourself Concerns and Wellbeing (MYCaW) assesses patient-reported outcomes (PROM) (46). This measure has the space for participants to write down two concerns they want help with, and then, the concerns are rated for severity using a 6-point Likert scale ranging from ‘not bothering me at all (0)’ to ‘bothers me greatly (6)’. A person’s wellbeing is also rated using the same 6-point Likert scale. There are two open-ended questions that ask if anything else important is happening in the person’s life and what has been most important about the treatment they received.

Researchers in the iCHOOSE Study are developing a core outcome set for stillbirth care research (55, 56), and this will guide future outcome measures for stillbirth research. The validated self-reported questionnaires evaluating worry and anxiety have been used together in previous research as outcome measures in stillbirth research (57) and influenced our use of these outcome measures. Worry, anxiety and stress have ‘intertwined behavioural and neural underpinnings’ (58); however, they have important unique characteristics that impact the way that they are experienced. Evaluating worry, stress and anxiety allows the researchers to capture the potential varied psychological states that can occur throughout a pregnancy after stillbirth especially when milestone events are passed without bad news/harm (e.g. gestation of previous loss, genetic and or anomaly scans).

A post-intervention qualitative questionnaire designed by the study researchers was based on a questionnaire used in previous stillbirth and massage research (36) (see Additional file 4) and assesses treatment utilisation and acceptability (Q.A-D), the appropriateness and suitability of the validated outcome measures the study used (Q.E), participant’s views of the study benefits (Q.G-L) and the participants’ perception of the study intervention to be a support for pregnant individuals via a massage intervention (Q.M).

An in-depth interview is intended to collect rich descriptions of participant experiences of massage and identify important aspects of care as well as provide information on the acceptability of the study intervention. Twenty study participants will be interviewed using a semi-structured approach by the study PI. The interviews will be conducted online via Zoom, recorded and transcribed verbatim by Cockatoo AI transcription services (59).

Data collection

Some women may give birth before receiving their allocated treatments. Intervention delivery and data collection will cease for participants who give birth prior to completing the full study protocol, and collected data will be kept within the data set. Attempts will be made to collect the final quantitative and qualitative data from participants who fall into this category. The inclusion criteria for participants to be no more than 30 weeks of gestation (Table  1 ) are in place to minimise participants giving birth prior to completing the full study protocol. Participants who give birth prior to completing the full study protocol will be emailed inviting them to complete the post-intervention online questionnaires and informing them that they are able to participate in the in-depth interview (if applicable) if they desire. A follow-up email will be sent 2 weeks after the first email.

Data analysis

Analysis of quantitative data.

The participant’s demographic and clinical characteristics will be summarised using descriptive statistics. Summary statistics of the observational clinical treatment data (e.g. worry, anxiety, etc.) will be reported and an appropriate matched test will be used to determine significant differences between baseline and post-final treatment scores (e.g. paired t -test if data are normally distributed, Wilcoxon matched-pairs test if not).

Three treatment effect modifiers have been identified: (1) the amount of other support services utilised, (2) passing the gestational age during the study intervention that they experienced their stillbirth, (3) the time between the stillbirth and this pregnancy, and (4) the number of study intervention treatments received. The correlational analysis will be used to determine associations between observational treatment outcomes and support services utilized (e.g. talk therapy (psychology/counselling), peer support etc.). Appropriate comparative tests will be used to determine whether there is a relationship between observed treatment outcomes and gestation of stillbirth, the duration of the interpregnancy interval, and the number of treatments administered.

Qualitative data analysis

The post-intervention questionnaire data and the open-ended MYCaW answers will undergo summative content analysis. The in-depth interview data will be analysed using thematic inductive analysis with the interview schedule modified to include emerging themes. Two researchers will immerse and familiarise themselves with the data to ascertain and identify the key concepts (60) The emerging themes will be discussed with members of the wider research team until a consensus is reached. An inductive approach will be utilised for analysis, as this method enables themes to be derived directly from the text data rather than being preconceived (60). To address biases in qualitative research methodology the researchers will provide a reflexivity statement acknowledging their role in the research and will undertake respondent validation which checks with participants to see if the findings still ring true.

There is considerable debate about what data saturation is and how it is determined depending on the qualitative research approach used and the data saturation model used (61–64). Instead of using the concept of data saturation as a determinate of qualitative data collection, the final sample size of our qualitative data collection will be shaped ‘by the adequacy (richness, complexity) of the data for addressing the research question’ (61) and the narrative approach that the final data will be presented as part of a mixed-methods methodology.

Mixed-methods analysis

A convergent parallel mixed-methods methodology is utilised for the HOPES study. After separate analyses of the quantitative and qualitative data, the data will be compared and related for areas of convergence or divergence and then interpreted. A narrative and joint display approach will be used to integrate data, and the fit of data integration will be reported. A joint display is a visual framework and is intended to help compare findings from the qualitative and quantitative data and generate meta-inferences (65). A narrative approach is used to further develop and provide context for quantitative findings (66). The more vernacular feel of the narrative approach helps participating individuals ‘hear’ their voice in the findings.

Dissemination of the findings

All participants will be offered a draft copy of any papers published as well as an easy-to-understand summary of the study findings. This will be emailed to the participants. A summary of the results and a copy of all the papers will be submitted to the funder.

Specialised antenatal care for women experiencing pregnancy after a stillbirth is recommended by researchers and clinicians (4, 13, 67); however, there is a lack of direct evidence on what specific interventions or approaches to care might benefit women experiencing pregnancy after stillbirth (15). The current research will begin to address the lack of high-grade evidence for supportive care options for individuals pregnant after a stillbirth. This protocol for a prospective feasibility study aims to measure the acceptability and preliminary effects of a massage intervention for women who are pregnant after a stillbirth or TMFR. The overarching intent of this work is to establish the feasibility of the protocol presented here for employment in a larger-scale intervention study. Progression criteria are listed below and use a Red-Amber-Green system based on previous feasibility work (14, 68) covering both qualitative and quantitative criteria (69).

Feasibility success criteria

Progression criterion: willingness of participants to use massage as a support option.

Participants will be considered willing to consider using massage as a support option based on the recruitment of the sample.

Red: Recruit < 60% of the required sample.

Amber: Recruit 60–80% of the required sample.

Green: Recruitment > 80%.

Progression criterion: Retention in the study

Participants will be considered retained in the study if they complete all the treatment intervention visits.

Red: Retain < 60% of participants in the study.

Amber: Retain 60–80% of participants in the study.

Green: Retain > 80% of participants in the study.

Progression criteria: Intervention experienced as supportive

Participants will be considered to have found the intervention supportive if they answer yes to either ‘I feel very supported’ or ‘I feel extremely supported’ for Q.M in the post-intervention qualitative questionnaire.

Red: Supportive < 60% of participants at the requisite level described above.

Amber: Supportive 60–80% of participants at the requisite level described above.

Green: Supportive > 80% of participants at the requisite level described above.

Progression criteria: Intervention implementation and experience

Assess the feasibility of delivering and experiencing the intervention in a way that is acceptable to pregnant women who have previously experienced stillbirth/TFMR.

Red: Delivery and experience of intervention judged possibly feasible by qualitative data.

Amber: Delivery and experience of intervention judged feasible by qualitative data.

Green: Delivery and experience of intervention judged strongly feasible by qualitative data.

The methodology and design of the HOPES study use a model of integrating the participant as the expert on their needs for support and care during pregnancy after a stillbirth and reflects elements considered essential by families who have experienced a subsequent pregnancy after stillbirth. Specifically, Gower et al. found a high-quality excellent care involved affirming and recognising pregnant people’s anxiety while being kind and empathetic and incorporating individualized emotionally supportive care (4). This is reflected in the study design via the conceptual approach that massage is just more than the hands-on application of treatment and via the participants being able to make therapeutic decisions about what they require addressing at each treatment session as well as determining the timing of their treatments to meet their unique needs. A participant-centred, participant-led approach in clinical practice has been shown to improve satisfaction with the care experience and improve health outcomes (70, 71).

A key study strength is the application of the intervention in a real-world setting. This allows the researchers to analyse the potential effects and benefits of massage in a setting relatable to how massage is currently provided. This is important in translating any findings into practice and allowing commercial massage therapists to incorporate this type of massage into their practice. The sample selection bias due to participants self-selecting to participate in the study is a study limitation, but this sample is reflective of the clinical practice where consumers self-select to seek massage treatment. The lack of a comparison group may be considered a limitation of this study; however, the challenges of deploying an individual randomised-controlled trial design are recognised among researchers active in care in pregnancy after loss (15).

This paper outlines the research protocol for the HOPES study, a mixed-methods feasibility trial of a massage intervention for women who are pregnant after a stillbirth or TMFR. The overarching intent of this work is to establish the feasibility of the HOPES study protocol for employment in a larger-scale effectiveness trial.

Availability of data and materials

Not applicable.

Mothers & Babies [Internet]. 2018 [cited 7 Feb 2018]. Available from: https://www.aihw.gov.au/reports-statistics/population-groups/mothers-babies/about .

Mothers & Babies [Internet]. 2020 [cited 15 Novr 2020]. Available from: https://www.aihw.gov.au/reports/mothers-babies/stillbirths-and-neonatal-deaths-in-australia/contents/overview-of-perinatal-deaths .

PWC. The economic impacts of stillbirth in Australia. 2016.

Gower S, Luddington J, Khosa D, Thaivalappil A, Papadopoulos A. Subsequent pregnancy after stillbirth: a qualitative narrative analysis of Canadian families’ experiences. BMC Pregnancy Childbirth. 2023;23(1):208.

Article   PubMed   PubMed Central   Google Scholar  

Mills TA, Ricklesford C, Cooke A, Heazell AE, Whitworth M, Lavender T. Parents’ experiences and expectations of care in pregnancy after stillbirth or neonatal death: a metasynthesis. BJOG. 2014;121(8):943–50.

Article   CAS   PubMed   Google Scholar  

Meaney S, Everard CM, Gallagher S, O’Donoghue K. Parents’ concerns about future pregnancy after stillbirth: a qualitative study. Health Expect. 2017;20(4):555–62.

Article   PubMed   Google Scholar  

Ellis A, Chebsey C, Storey C, Bradley S, Jackson S, Flenady V, et al. Systematic review to understand and improve care after stillbirth: a review of parents’ and healthcare professionals’ experiences. BMC Pregnancy Childbirth. 2016;16:16.

Cacciatore J. Psychological effects of stillbirth. Semin Fetal Neonatal Med. 2013;18(2):76–82.

Wojcieszek AM, Shepherd E, Middleton P, Lassi ZS, Wilson T, Murphy MM, et al. Care prior to and during subsequent pregnancies following stillbirth for improving outcomes. Cochrane Database Syst Rev. 2018;12(12):Cd012203.

Turton P, Hughes P, Evans CD, Fainman D. Incidence, correlates and predictors of post-traumatic stress disorder in the pregnancy after stillbirth. Br J Psychiatry. 2001;178:556–60.

Gravensteen IK, Jacobsen EM, Sandset PM, Helgadottir LB, Rådestad I, Sandvik L, Ekeberg Ø. Anxiety, depression and relationship satisfaction in the pregnancy following stillbirth and after the birth of a live-born baby: a prospective study. BMC Pregnancy Childbirth. 2018;18(1):41.

Hughes PM, Turton P, Evans CD. Stillbirth as risk factor for depression and anxiety in the subsequent pregnancy: cohort study. BMJ. 1999;318(7200):1721–4.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ladhani NNN, Fockler ME, Stephens L, Barrett JFR, Heazell AEP. No. 369-Management of pregnancy subsequent to stillbirth. J Obstet Gynaecol Can. 2018;40(12):1669–83.

Mills TA, Roberts SA, Camacho E, Heazell AEP, Massey RN, Melvin C, et al. Better maternity care pathways in pregnancies after stillbirth or neonatal death: a feasibility study. BMC Pregnancy Childbirth. 2022;22(1):634.

Wojcieszek AM, Heazell AEP, Middleton P, Ellwood D, Silver RM, Flenady V. Research priorities and potential methodologies to inform care in subsequent pregnancies following stillbirth: a web-based survey of healthcare professionals, researchers and advocates. BMJ Open. 2019;9(6): e028735.

Flenady V, Gardener G, Ellwood D, Coory M, Weller M, Warrilow KA, et al. My Baby’s Movements: a stepped-wedge cluster-randomised controlled trial of a fetal movement awareness intervention to reduce stillbirths. BJOG. 2022;129(1):29–41.

Norman JE, Heazell AEP, Rodriguez A, Weir CJ, Stock SJE, Calderwood CJ, et al. Awareness of fetal movements and care package to reduce fetal mortality (AFFIRM): a stepped wedge, cluster-randomised trial. Lancet. 2018;392(10158):1629–38.

Guo PP, Fan SL, Li P, Zhang XH, Liu N, Wang J, et al. The effectiveness of massage on peri-operative anxiety in adults: a meta-analysis of randomized controlled trials and controlled clinical trials. Complement Ther Clin Pract. 2020;41: 101240.

Smith CA, Shewamene Z, Galbally M, Schmied V, Dahlen H. The effect of complementary medicines and therapies on maternal anxiety and depression in pregnancy: a systematic review and meta-analysis. J Affect Disord. 2019;245:428–39.

Field T, Deeds O, Diego M, Hernandez-Reif M, Gauler A, Sullivan S, et al. Benefits of combining massage therapy with group interpersonal psychotherapy in prenatally depressed women. J Bodyw Mov Ther. 2009;13(4):297–303.

Field T, Diego M, Hernandez-Reif M, Medina L, Delgado J, Hernandez A. Yoga and massage therapy reduce prenatal depression and prematurity. J Bodyw Mov Ther. 2012;16(2):204–9.

Field T, Diego MA, Hernandez-Reif M, Schanberg S, Kuhn C. Massage therapy effects on depressed pregnant women. J Psychosom Obstet Gynaecol. 2004;25(2):115–22.

Field T, Figueiredo B, Hernandez-Reif M, Diego M, Deeds O, Ascencio A. Massage therapy reduces pain in pregnant women, alleviates prenatal depression in both parents and improves their relationships. J Bodyw Mov Ther. 2008;12(2):146–50.

Field T, Hernandez-Reif M, Hart S, Theakston H, Schanberg S, Kuhn C. Pregnant women benefit from massage therapy. J Psychosom Obstet Gynaecol. 1999;20(1):31–8.

Hall HG, Cant R, Munk N, Carr B, Tremayne A, Weller C, et al. The effectiveness of massage for reducing pregnant women’s anxiety and depression; systematic review and meta-analysis. Midwifery. 2020;90: 102818.

Davis L, Hanson B, Gilliam S. Pilot study of the effects of mixed light touch manual therapies on active duty soldiers with chronic post-traumatic stress disorder and injury to the head. J Bodyw Mov Ther. 2016;20(1):42–51.

Price C. Body-oriented therapy in recovery from child sexual abuse: an efficacy study. Altern Ther Health Med. 2005;11(5):46–57.

PubMed   PubMed Central   Google Scholar  

Yuan SL, Matsutani LA, Marques AP. Effectiveness of different styles of massage therapy in fibromyalgia: a systematic review and meta-analysis. Man Ther. 2015;20(2):257–64.

Letourneau NL, Dennis CL, Cosic N, Linder J. The effect of perinatal depression treatment for mothers on parenting and child development: a systematic review. Depress Anxiety. 2017;34(10):928–66.

Field T, Diego M, Hernandez-Reif M, Deeds O, Figueiredo B. Pregnancy massage reduces prematurity, low birthweight and postpartum depression. Infant Behav Dev. 2009;32(4):454–60.

Mueller SM, Grunwald M. Effects, side effects and contraindications of relaxation massage during pregnancy: a systematic review of randomized controlled trials. J Clin Med. 2021;10(16):3485.

Pachtman Shetty SL, Fogarty S. Massage during pregnancy and postpartum. Clinical Obstetrics and Gynecology. 2021;64(3).

Fogarty S. A role for massage after antenatal or neonatal loss: evaluations from a community program. Advances in Integrative Medicine. 2021;8(2):129–35.

Article   Google Scholar  

Cronfalk BS, Ternestedt B-M, Strang P. Soft tissue massage: early intervention for relatives whose family members died in palliative cancer care. J Clin Nurs. 2010;19:1040–8.

Bishop MD, Torres-Cueco R, Gay CW, Lluch-Girbés E, Beneciuk JM, Bialosky JE. What effect can manual therapy have on a patient’s pain experience? Pain Manag. 2015;5(6):455–64.

Fogarty S. The use of massage to support emotional and physical health in a pregnancy after stillbirth: a case study. International Journal of Therapeutic Massage & Bodywork. 2021;In press.

SANDS miscarriage, stillbirth & newborn death support: Red Nose; 2021 [Available from: https://www.sands.org.au .

Malt Creative. Red Nose: Red Nose; 2023 [Available from: https://rednose.org.au .

Burden C, Bradley S, Storey C, Ellis A, Heazell AEP, Downe S, et al. From grief, guilt pain and stigma to hope and pride – a systematic review and meta-analysis of mixed-method research of the psychosocial impact of stillbirth. BMC Pregnancy Childbirth. 2016;16(1):9.

Westby CL, Erlandsen AR, Nilsen SA, Visted E, Thimm JC. Depression, anxiety, PTSD, and OCD after stillbirth: a systematic review. BMC Pregnancy Childbirth. 2021;21(1):782.

Blackmore ER, Côté-Arsenault D, Evans J, Glover V, Golding J, O’Connor TG, Tang W. Previous prenatal loss as a predictor of perinatal depression and anxiety. Br J Psychiatry. 2011;198(5):373–8.

Gravensteen IK, Jacobsen E-M, Sandset PM, Helgadottir LB, Rådestad I, Sandvik L, Ekeberg Ø. Anxiety, depression and relationship satisfaction in the pregnancy following stillbirth and after the birth of a live-born baby: a prospective study. BMC Pregnancy Childbirth. 2018;18(1):41.

Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res. 2016;25(3):1057–73.

Crouch M, McKenzie H. The logic of small samples in interview-based qualitative research. Soc Sci Inf. 2006;45(4):483–99.

Gois C, Dias VV, Raposo JF, do Carmo I, Barbosa A. Vulnerability to stress, anxiety and depressive symptoms and metabolic control in type 2 diabetes. BMC Res Notes. 2012;5:271.

Paterson C, Thomas K, Manasse A, Cooke H, Peace G. Measure Yourself Concerns and Wellbeing (MYCaW): an individualised questionnaire for evaluating outcome in cancer support care that includes complementary therapies. Complement Ther Med. 2007;15(1):38–45.

Fogarty S, McInerney C, Stuart C, Hay P. The side effects and mother or child related physical harm from massage during pregnancy and the postpartum period: An observational study. Complement Ther Med. 2019;42:89–94.

Green JM, Kafetsios K, Statham HE, Snowdon CM. Factor structure, validity and reliability of the Cambridge Worry Scale in a pregnant population. J Health Psychol. 2003;8(6):753–64.

Mortazavi F, Mehrabadi M, KiaeeTabar R. Pregnant women’s well-being and worry during the COVID-19 pandemic: a cross-sectional study. BMC Pregnancy Childbirth. 2021;21(1):59.

Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

Hamilton JG, Lobel M. Types, patterns, and predictors of coping with stress during pregnancy: examination of the Revised Prenatal Coping Inventory in a diverse sample. J Psychosom Obstet Gynaecol. 2008;29(2):97–104.

Lev EL, Eller LS, Kolassa J, Gejerman G, Colella J, Lane P, et al. Exploratory factor analysis: strategies used by patients to promote health. World J Urol. 2007;25(1):87–93.

Nielsen MG, Ørnbøl E, Vestergaard M, Bech P, Larsen FB, Lasgaard M, Christensen KS. The construct validity of the Perceived Stress Scale. J Psychosom Res. 2016;84:22–30.

Mercer SW, Maxwell M, Heaney D, Watt GC. The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004;21(6):699–705.

Bakhbakhi D, Fraser A, Siasakos D, Hinton L, Davies A, Merriel A, et al. Protocol for the development of a core outcome set for stillbirth care research (iCHOOSE Study). BMJ Open. 2022;12(2): e056629.

Bakhbakhi D, Siassakos D, Davies A, Merriel A, Barnard K, Stead E, et al. Interventions, outcomes and outcome measurement instruments in stillbirth care research: a systematic review to inform the development of a core outcome set. BJOG: An International Journal of Obstetrics & Gynaecology. 2023;130(6):560–76.

Thomas S, Stephens L, Mills TA, Hughes C, Kerby A, Smith DM, Heazell AEP. Measures of anxiety, depression and stress in the antenatal and perinatal period following a stillbirth or neonatal death: a multicentre cohort study. BMC Pregnancy Childbirth. 2021;21(1):818.

Daviu N, Bruchas MR, Moghaddam B, Sandi C, Beyeler A. Neurobiological links between stress and anxiety. Neurobiol Stress. 2019;11: 100191.

Cockatoo. Transcription powered by AI 2023 [Available from: https://www.cockatoo.com/?gad=1&gclid=Cj0KCQjwz8emBhDrARIsANNJjS7a9s_5QLZbxfPw3CJQcbDIMgaX1L12Fpi1FNjWiS6wmXeHhjdf28caAtEFEALw_wcB .

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.

Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health. 2021;13(2):201–16.

Fofana F, Bazeley P, Regnault A. Applying a mixed methods design to test saturation for qualitative data in health outcomes research. PLoS ONE. 2020;15(6): e0234898.

Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2016;26(13):1753–60.

Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2018;52(4):1893–907.

McCrudden MT, Marchand G, Schutz PA. Joint displays for mixed methods research in psychology. Methods in Psychology. 2021;5: 100067.

Doran P, Burden S, Shryane N. Integration of narratives into mixed methods research: an example from a study on the value of social support to older people with cancer. J Mixed Methods Res. 2021;16(4):418–37.

Graham N, Stephens L, Heazell AEP. Care in pregnancies subsequent to stillbirth or perinatal death. Obstet Gynaecol. 2021;23(1):48–59.

Huberty J, Matthews J, Leiferman J, Cacciatore J, Gold KJ. A study protocol of a three-group randomized feasibility trial of an online yoga intervention for mothers after stillbirth (The Mindful Health Study). Pilot and Feasibility Studies. 2017;4(1):12.

Mellor K, Albury C, Dutton SJ, Eldridge S, Hopewell S. Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. Pilot and Feasibility Studies. 2023;9(1):59.

Krist AH, Tong ST, Aycock RA, Longo DR. Engaging patients in decision-making and behavior change to promote prevention. Stud Health Technol Inform. 2017;240:284–302.

Australian Commission on Safety and Quality in Health Care. Patient- centred care: improving quality and safety through partnerships with patients and consumers. Sydney; 2011.

Download references

Acknowledgements

Not applicable

The study is funded by the 2022 Research Support Grant from The Massage Therapy Foundation awarded to SF. The funders had no role in the design of this trial and will not have any role during its execution, analyses, data interpretation or decision to submit results.

Author information

Authors and affiliations.

School of Medicine, Western Sydney University, Locked Bag 1797, Penrith, NSW, 2751, Australia

Sarah Fogarty

Translational Health Research Institute, Western Sydney University, Locked Bag 1797, Penrith, NSW, 2751, Australia

Sarah Fogarty, Alexander E. P. Heazell & Phillipa Hay

School of Medical Sciences, Maternal and Fetal Health Research Centre, University of Manchester, Manchester, UK

Alexander E. P. Heazell

Department of Obstetrics, Saint Mary’s Hospital, Manchester University NHS Foundation Trust, Manchester, UK

School of Health & Human Sciences, Indiana University, Indianapolis, USA

Australian Research Centre in Complementary and Integrative Medicine (ARCCIM), Fellow and Visiting Faculty of Health, University of Technology Sydney, Massage & MyotherapyAustralia, Sydney, Australia

Mental Health Services, SWSLHD, Campbelltown Hospital, Campbelltown, NSW, Australia

Phillipa Hay

You can also search for this author in PubMed   Google Scholar

Contributions

SF, PH, NM and AH contributed to all phases of the protocol writing (initial idea, research question, design and assessments). SF completed the ethics proposal and amendments. SF and NM developed the massage treatment protocol. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sarah Fogarty .

Ethics declarations

See the section ‘ Ethics approval and consent to participate ’ within the main text. Human ethics approval for the study was granted by the Ethics Committee of Western Sydney University (No: H15261).

Consent for publication

Competing interests.

SF is a practising massage therapist. NM is a full-time academic, long-time consultant and mentor to SF, who receives no financial compensation for her effort on this work and is a trustee of The Massage Therapy Foundation. It is not expected that the study findings will yield any financial gain for SF or NM. NM’s role at the Massage Therapy Foundation has no bearing on the study implementation or results, and findings from this study will be submitted for publication regardless of outcome. PH is a consultant to Takeda Pharmaceuticals.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Fogarty, S., Heazell, A.E.P., Munk, N. et al. Swedish massage as an adjunct approach to Help suppOrt individuals Pregnant after Experiencing a prior Stillbirth (HOPES): a convergent parallel mixed-methods single-arm feasibility trial protocol. Pilot Feasibility Stud 10 , 67 (2024). https://doi.org/10.1186/s40814-024-01499-z

Download citation

Received : 20 September 2023

Accepted : 18 April 2024

Published : 30 April 2024

DOI : https://doi.org/10.1186/s40814-024-01499-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mixed-methods
• Termination for medical reasons
• Pregnancy after loss

Pilot and Feasibility Studies

ISSN: 2055-5784

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]