MomsTalkShots, tailored educational app, improves vaccine attitudes: a randomized controlled trial
BMC Public Health volume 22, Article number: 2134 (2022)
Many pregnant women and parents have concerns about vaccines. This analysis examined the impact of MomsTalkShots, an individually tailored educational application, on vaccine attitudes of pregnant women and mothers.
MomsTalkShots was the patient-level component of a multi-level intervention to improve maternal and infant vaccine uptake that also included provider- and practice-level interventions. The impact of these interventions was studied using a two-by-two factorial design, randomizing at both the patient- and the practice-level. Study staff recruited pregnant women from a diverse set of prenatal care practices in Colorado and Georgia between June 2017 and July 2018. All participants (n = 2087) received a baseline survey of maternal and infant vaccine intentions and attitudes, and two follow-up surveys at least 1 month and 1 year after their infant’s birth, respectively. Half of participants (n = 1041) were randomly assigned to receive educational videos through MomsTalkShots, algorithmically tailored to their vaccine intentions, attitudes, and demographics. Since the practice/provider intervention did not appear impactful, this analysis focused on MomsTalkShots regardless of the practice/provider intervention.
By 1 month post-birth, MomsTalkShots increased perceived risk of maternal influenza disease (61% among MomsTalkShots recipients vs 55% among controls; Odds Ratio: 1.61, 95% Confidence Interval: 1.23–2.09), confidence in influenza vaccine efficacy (73% vs 63%; OR: 1.97, 95%CI: 1.47–2.65), and perceived vaccine knowledge (55% vs 48%; OR: 1.39, 95%CI: 1.13–1.72). Among those intending not to vaccinate at baseline, MomsTalkShots increased perceived risk of maternal influenza disease (38% vs 32%; OR: 2.07, 95%CI: 1.15–3.71) and confidence in influenza vaccine efficacy (44% vs 28%; OR: 2.62, 95%CI: 1.46–4.69).
By 1 year post-birth, MomsTalkShots increased perceived vaccine knowledge (62% vs 50%; OR: 1.74, 95%CI: 1.36–2.24) and trust in vaccine information from obstetricians and pediatricians (64% vs 55%; OR: 1.53, 95%CI: 1.17–2.00). Among those uncertain about vaccinating at baseline, MomsTalkShots increased perceived vaccine knowledge (47% vs 12%; OR: 6.89, 95%CI: 1.52–31.25) and reduced infant vaccine safety concerns (71% vs 91%; OR: 0.24, 95%CI: 0.06–0.98).
MomsTalkShots improved pregnant women’s and mothers’ knowledge and perceptions of maternal and infant vaccines and the diseases they prevent, and offers a scalable tool to address vaccine hesitancy.
Registered at Clinicaltrials.gov on 13/09/2016 (registration number: NCT02898688).
Only 55% of pregnant women in the United States (US) received influenza vaccination during the 2020–2021 influenza season . Though this exceeded the 50% coverage among all adults,  it fell well short of the Healthy People 2020 goal of 80% coverage among pregnant women . Only 54% received tetanus, diphtheria, and acellular pertussis (Tdap) vaccination .
Many pregnant women [4,5,6,7,8,9,10,11] and parents [12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28] have concerns about vaccines. Vaccine knowledge, attitudes and beliefs (KABs) are strongly associated with vaccine behavior . Parents often start making vaccine decisions for their children during or soon after their first pregnancy, especially those with negative attitudes toward vaccines . The prenatal period is thus an optimal time to educate about both maternal and infant vaccines, potentially reaching an undecided audience before overexposure to misinformation leads to rigid misperceptions .
However, many pregnant women do not receive adequate information about vaccines directly from their prenatal care providers, instead relying on internet searches and their social networks . Healthcare providers are busy, and discussions with vaccine hesitant patients can be long and burdensome . Many prenatal care providers do not have the same level of experience and training discussing vaccination with their patients as pediatricians . Tools are needed to address maternal and infant vaccine concerns and ease this burden from providers, but messaging must be tailored, [34, 35] as vaccine KABs and acceptance have been shown to differ substantially by gender, education, socioeconomic status, ethnicity and race, [36,37,38,39,40,41,42,43,44,45] including among pregnant women .
As one component of a multi-level intervention to encourage immunization of pregnant women and their infants, [47, 48] we designed MomsTalkShots, an individually tailored educational application (app) . Among a sample of pregnant women from Georgia and Colorado, MomsTalkShots substantially increased influenza vaccine uptake among women who initially did not intend to vaccinate during pregnancy. No significant increase in Tdap vaccine uptake was found though baseline Tdap vaccine coverage was very high across all practices . The main objective of this analysis was to evaluate the impact of MomsTalkShots on vaccine KABs.
Materials and methods
The MomsTalkShots app (described in more detail elsewhere)  was designed as a website to be accessible via multiple internet browsers on smartphones, tablets, and computers. The app begins with registration, then administers a survey, then immediately after survey completion provides educational videos that are algorithmically responsive to its users’ vaccine intentions, KABs, and demographics . The videos incorporate introductions and conclusions from obstetricians and pediatricians of different races/ethnicities with narrated animation to communicate messages in an interesting and engaging manner. The videos were designed based on approaches shown to be effective in training healthcare providers to improve their vaccine discussions with patients: for patients already intending to vaccinate, taking a presumptive approach; and for patients with concerns, establishing empathy, then carefully addressing the concerns within the context of the risk of disease, the benefits of vaccination, and the ability to protect through vaccinating [50,51,52,53,54]. Videos covered both maternal and infant vaccination, though videos provided to pregnant women focused more on the former and videos provided to new mothers focused more on the latter. Videos were also available in a gallery for rewatching upon logging in between survey timepoints. A feature allowing pregnant women to refer the app to their close contacts was evaluated elsewhere and was shown to increase influenza vaccine uptake among family and friends to cocoon the infant [55, 56].
MomsTalkShots was the patient-level component of a multi-level intervention to improve maternal and infant vaccine uptake that also included provider- and practice-level interventions, described in more detail elsewhere [47, 48]. The impact of these interventions was studied using a two-by-two factorial design, randomizing at both the patient- and the practice-level (provider-level interventions were implemented and analysed in tandem with practice-level interventions). The practice-level intervention implemented an adaptation of the Centers for Disease Control and Prevention (CDC) Assessment, Feedback, Incentives and Exchange (AFIX) model,  and the provider-level interventions included provision of the Continuing Medical Education (CME) module VaxChat  and the book The Clinician’s Vaccine Safety Resource Guide . The target sample size for the overall trial of 1896 was calculated using PASS 11 (NCSS, LLC, Kaysville UT), assuming a cluster randomized design with intra-class correlation coefficient of 0.01 (based on preliminary data from a previous pilot study),  to provide a power of 80% (α = 0.05) to see a doubling in maternal Tdap vaccinations (not for the purpose of this analysis) .
Study staff recruited pregnant women from waiting rooms of a geographically and socio-demographically diverse set of prenatal care practices in Colorado and Georgia between June 2017 and July 2018 and followed them through March 2019. Practices were chosen to capture diversity in patient demographics, urbanicity, provider types, and practice size. Eleven of the 22 participating practices were assigned to receive the practice- and provider-level interventions (via covariate constrained cluster randomization) . Patient eligibility criteria included: between 18 and 50 years of age, English-speaking, and gestational age of 8–26 weeks (since Tdap is recommended at 27–36 weeks gestation) . Upon enrollment, study staff lent participants an electronic tablet to complete the baseline survey via the app while in the waiting room. Two follow-up surveys were made available to all participants via the app approximately 1 month and 1 year after their infant’s birth, respectively, to complete using their device of choice at home. A $20 incentive was provided for completion of each survey. Further detail and data on recruitment is reported elsewhere .
All participants received three surveys, but only the half randomly assigned to receive the patient-level intervention subsequently received the tailored educational videos (MomsTalkShots) immediately after the baseline and initial follow-up surveys. We used SAS PROC PLAN to generate patient-level randomization schedules for each of the 22 participating practices, using block randomization with a block size of 8; these randomization sequences were then programmed into the app such that participants were randomized upon registration, blinded to study investigators.
Surveys used multiple choice questions to assess vaccine intentions, and Likert scale statements to assess latent vaccine KAB constructs, including: confidence in vaccine safety and efficacy, perceived susceptibility to and severity of vaccine-preventable diseases, self-efficacy (one’s belief in their ability to execute behaviors necessary to reach specific goals),  descriptive (what people do) and injunctive (what people approve) norms,  perceived vaccine knowledge, and trust in sources of vaccine information. These constructs were chosen after review of relevant behavioral models and scales, [8, 64] and multiple survey items were dedicated to each construct. Likert scale response options included: strongly agree, agree, disagree, and strongly disagree; knowledge and trust statements also included “don’t know”; and trust statements regarding pediatricians and naturopathic/chiropractic doctors also included “I don’t have a pediatrician yet” and “I don’t see this type of doctor,” respectively. Specific vaccine safety concern statements were administered only to those who expressed a lack of confidence in vaccine safety (to avoid creating new concerns).
Likert scale responses to survey items were encoded as follows: 1 - strongly disagree; 2 - disagree; 3 - don’t know; 4 - agree; 5 - strongly agree. Those who agreed (or strongly agreed) with survey items expressing confidence in vaccine safety were encoded to have disagreed (or strongly disagreed) with survey items expressing specific vaccine safety concerns. Those who had not (yet) seen pediatricians and/or naturopathic/chiropractic doctors were treated as missing for these statements.
Summary scores for constructs combined the scores from their constituent statements. Only constructs with multiple statements assessed at two or more timepoints were analysed. All scores were standardized by dividing into the maximum score then multiplying by 100. This standardization allowed comparison of construct summary scores with different numbers of contributing statements.
Dichotomous variables were encoded for each of the survey items as follows: 1 - agree or strongly agree; 0 - disagree, strongly disagree, or don’t know. Dichotomous construct summary scores were created by calculating the mean of each scaled continuous construct summary score and encoding as above the mean (“high”) or below the mean (“low”).
Analysis was guided by the statistical analysis plan documented in the study protocol and recorded on clinicaltrials.gov. Multilevel mixed-effects linear regressions were performed for scaled statement and construct scores, with a random intercept for the clinic. Interim and final time point scores were each regressed on their corresponding baseline score, to control for potential differences in baseline scores between groups. Dummy variables were initially included for MomsTalkShots, the practice/provider intervention, and their product (to test for potential interaction). However, the interaction term was found not to be statistically significant, so the model was reparametrized without a dummy variable for interaction. Regressions were stratified by baseline intention to vaccinate. Dichotomous/logistic analysis otherwise emulating the above methodology was performed to increase interpretability. Statistical significance was considered p < 0.05.
Per protocol, because our surveys included so many individual items, we focused our analysis on constructs, to reduce redundancy and the chances of type 1 error. However, individual survey statements were also analysed, to provide additional insight into the individual components driving changes in constructs. The analyses of individual survey statements are reported as appendices.
Since the practice/provider intervention did not appear impactful, and the impact of MomsTalkShots did not significantly differ between intervention and non-intervention practices, we focused on MomsTalkShots regardless of the practice/provider intervention.
The results of the dichotomous analysis (Additional file 1: Appendices 1–2) were highly consistent with the results of the continuous analysis (Additional file 1: Appendices 3–4), aside from small differences in significance due to loss of power, so we present the results of the dichotomous analysis for greater interpretability.
Enrollment and follow-up
Of the 3904 pregnant women found to be eligible after screening, 1391 declined to participate, and 303 did not finish enrollment. Ultimately 2087 enrolled pregnant women were randomized and provided enough data to contribute to this analysis. Informed consent was obtained from all participants.
Nearly three quarters of the participating pregnant women (n = 1524; 73%) completed the initial follow-up survey (at least 1 month after their infant’s birth), and over half (n = 1117; 54%) completed the final follow-up survey (at least 1 year after their infant’s birth) (Additional file 1: Appendix 7). Follow-up survey completion did not differ by intervention group, but was consistently lower among women not intending to vaccinate at baseline (Additional file 1: Appendix 8).
Half of participating pregnant women were from Georgia, and half from Colorado. Nearly half (46%) were pregnant for the first time. About 85% provided their race/ethnicity and 83% their education: of these, 64% were White, 16% were Black and 11% were Hispanic, and 72% had at least an undergraduate degree, respectively. Sociodemographic characteristics were similar between study arms,  and are described in more detail elsewhere [46, 48, 65].
Baseline vaccine intentions
Over half (56%) of women intended to receive both influenza and Tdap vaccines during pregnancy, 16% intended to receive one but not the other, 14% intended to receive neither, and 13% were unsure; 81% intended for their baby to receive all recommended vaccines, 11% intended for their baby to receive some or no recommended vaccines, and 8% were unsure (Table 1). Baseline vaccine intentions were similar between study arms,  and are described in more detail elsewhere [46, 48, 65].
Herein we report statistically significant results among all participants and stratified by vaccine intent.
Effect of MomsTalkShots among all participants
One month after their infant’s birth, 61% of MomsTalkShots recipients had high perceived risk of maternal influenza disease, compared to 55% of controls (Odds Ratio: 1.61, 95% Confidence Interval: 1.23–2.09); 73% had high confidence in maternal influenza vaccine efficacy, compared to 63% of controls (OR: 1.97, 95%CI: 1.47–2.65); and 55% had high perceived vaccine knowledge, compared to 48% of controls (OR: 1.39, 95%CI: 1.13–1.72) (Table 2). One year after their infant’s birth, 62% had high perceived vaccine knowledge, compared to 50% of controls (OR: 1.74, 95%CI: 1.36–2.24); and 64% had high trust in vaccine information from obstetricians and pediatricians, compared to 55% of controls (OR: 1.53, 95%CI: 1.17–2.00) (Table 3).
Effect of MomsTalkShots among participants intending to vaccinate
One month after their infant’s birth, 71% of MomsTalkShots recipients intending to vaccinate at baseline had high perceived risk of maternal influenza disease, compared to 64% of controls (OR: 1.57, 95%CI: 1.12–2.20); 86% had high confidence in maternal influenza vaccine efficacy, compared to 77% of controls (OR: 2.09, 95%CI: 1.36–3.22); and 59% had high perceived vaccine knowledge, compared to 53% of controls (OR: 1.36, 95%CI: 1.08–1.70) (Table 2). One year after their infant’s birth, 65% had high perceived vaccine knowledge, compared to 55% of controls (OR: 1.60, 95%CI: 1.22–2.10); and 70% had high trust in vaccine information from obstetricians and pediatricians, compared to 63% of controls (OR: 1.47, 95%CI: 1.09–1.97) (Table 3).
Effect of MomsTalkShots among participants intending not to vaccinate
One month after their infant’s birth, 38% of MomsTalkShots recipients intending not to vaccinate at baseline had high perceived risk of maternal influenza disease, compared to 32% of controls (OR: 2.07, 95%CI: 1.15–3.71); and 44% had high confidence in maternal influenza vaccine efficacy, compared to 28% of controls (OR: 2.62, 95%CI: 1.46–4.69) (Table 2).
Effect of MomsTalkShots among participants with uncertain vaccine intentions
One year after their infant’s birth, 47% of MomsTalkShots recipients with uncertain vaccine intentions at baseline had high perceived vaccine knowledge, compared to 12% of controls (OR: 6.89, 95%CI: 1.52–31.25); and 71% had high specific safety concerns for infant vaccines, compared to 91% of controls (OR: 0.24, 95%CI: 0.06–0.98) (Table 3).
Potential unintended effects
MomsTalkShots decreased perceived risk of infant pertussis: 1 month after their infant’s birth, 39% of MomsTalkShots recipients had high perceived risk of infant pertussis, compared to 42% of controls (OR: 0.75, 95%CI: 0.57–1.00) (Table 2). While MomsTalkShots decreased perceived susceptibility; it also increased perceived severity (Additional file 1: Appendix 1), and the reduction in perceived susceptibility was limited to those who received Tdap during pregnancy (Additional file 1: Appendix 5).
MomsTalkShots positively impacted pregnant women’s and mothers’ knowledge and perceptions of maternal and infant vaccines and the diseases they prevent. Among women initially intending not to vaccinate, MomsTalkShots increased perceived risk of maternal influenza disease and confidence in influenza vaccine efficacy. Among women with uncertain infant vaccine intentions, MomsTalkShots increased trust in obstetricians and pediatricians and substantially reduced safety concerns.
These findings correspond to our previous findings that MomsTalkShots substantially increased influenza vaccine uptake among pregnant women who initially did not intend to vaccinate during pregnancy . This contributes to the literature showing that changing attitudes can improve vaccine acceptance [29, 35]. These findings are notable, as vaccine education is typically ineffective unless implemented in tandem with other proven interventions (such as provider prompts and standing orders), [31, 66] with a few exceptions [67,68,69,70,71]. MomsTalkShots’ impact may be partly due to its ability to tailor its content to individual vaccine intentions, KABs, and demographics, further supporting such an approach to vaccine education [34, 35]. Also notable is the duration of impact; decreased safety concerns, increased perceived vaccine knowledge, and increased trust in obstetricians and pediatricians were found in women nearly a year after they received MomsTalkShots, despite evidence that didactic education often fades from memory after about a week .
The only unexpected construct association found was that MomsTalkShots decreased perceived risk of infant pertussis. Education strategies based on correcting vaccine misinformation or exposure to fear appeals also have the potential to backfire among those with strong preexisting levels of vaccine hesitancy, [73,74,75] and thus must be approached with caution and care. However, this decrease in perceived risk of infant pertussis was driven by a reduction in perceived susceptibility among women who received Tdap during pregnancy (Additional file 1: Appendix 5). Perceived severity increased after MomsTalkShots (Additional file 1: Appendix 1). So, women who received Tdap after learning how Tdap protects their infant from pertussis via MomsTalkShots correctly perceived a reduction in susceptibility of their infant to pertussis and correctly identified the increased severity of pertussis for infants.
Our study design included randomization at both the practice/provider and the patient levels. We hypothesized that both interventions might impact KAB constructs, and there might be an improved impact when combined. However, only the patient level intervention (MomsTalkShots) appeared to have an impact; the number of construct associations found with the practice/provider intervention was less than expected by chance alone and the interaction between practice/provider and patient interventions was non-significant. So we focused our per protocol analysis on constructs in the main analysis to reduce redundancy and the chances of type 1 error. There were many insignificant associations that we did not explicitly comment on in this manuscript; they may indicate MomsTalkShots did not impact certain vaccine perceptions, especially among those not intending to vaccinate at baseline. Although we analysed each KAB construct both as a continuous score and a dichotomous indicator of an above-average score, we focused exclusively on the dichotomous analysis in the Results and corresponding tables, despite continuous measures typically providing greater power and precision than dichotomous measures derived from them. However, we found the dichotomous analyses to be much more interpretable and thus more useful for the main text. We have presented both analyses fully in the Appendices for transparency; aside from small differences in significance due to loss of power, the results of the dichotomous analysis (Additional file 1: Appendices 1–2) were highly consistent with the results of the continuous analysis (Additional file 1: Appendices 3–4), justifying our approach. MomsTalkShots’ decrease in perceived risk of infant pertussis being driven by women who received Tdap during pregnancy illustrates the potential effect modification of vaccination, which was not included in our analytic approach beyond further exploration of an unexpected result. Small numbers of women intending their children to receive no recommended vaccines did not provide enough power to justify stratifying by this group alone, so it was combined with those who intended their children to receive some (but not all) recommended vaccines. Although this made our stratification less precise, it increased our power and simplified our analysis. Even with this combination, power was still limited, leading to many potential associations that were close to but not quite statistically significant at the prespecified p < 0.05 cutoff. This analysis focused on pregnant women and mothers and thus did not account for others who may be heavily involved in vaccine decisions for children, though data on partners, family and close friends of these pregnant women are published elsewhere [55, 56]. Our sample was comprised mostly of highly educated white women, despite efforts to recruit from a geographically and socio-demographically diverse set of prenatal care practices. Finally, loss to follow-up with differential rates by baseline intention to vaccinate may have biased our data, especially from the final follow-up at 1 year after birth, as those who intended to vaccinate were more likely both to follow-up and to interact positively with MomsTalkShots.
Evaluation of MomsTalkShots through a RCT has high internal validity and provides compelling efficacy data. Our study included recruitment by study coordinators in obstetric practices using financial incentives, whereas in a real-world setting, strategies would be needed to encourage pregnant women and mothers to use MomsTalkShots without these incentives. While scale-up is not a challenge from a technical perspective, dissemination and support from healthcare providers, public health authorities, and other partners would be critical. Other similar interventions do not offer this potential combination of effectiveness and scalability. In-person training of providers to improve their vaccine communication with patients has been shown to be effective, but scale-up would be cost- and time-intensive [52,53,54]. Several other educational vaccine apps and websites have been developed [70, 71, 76,77,78,79,80]. These include: ImmunizeCA, a smartphone app which helps Canadians manage their family’s immunizations by generating customized immunization schedules and reminder alerts for each family member ; ReadyVax, a smartphone app providing access to evidence-based vaccine information for providers and patients ; and HPV Vaccine: Same Way, Same Day, a smartphone app which teaches evidence-based vaccine recommendation practices including motivational interviewing skills using simulated role-play scenarios . However, only one – a web-based social media intervention during pregnancy – demonstrated a significant positive effect on vaccine uptake, and its scalability is a challenge due to its reliance on public interactions with vaccine experts, whose time is limited and expensive [70, 71]. MomsTalkShots is the only app or website that tailors information on vaccine attitudes, concerns and demographics. Further research is needed to identify characteristics beyond tailoring that make such apps and websites effective versus ineffective among various populations and settings.
The need for effective interventions to improve vaccine confidence and uptake has only increased since the conclusion of this study. The Coronavirus disease 2019 (COVID-19) pandemic has caused widespread morbidity and mortality, while disjointed government response has led to confusion, the unfortunate politicization of vaccination, and vaccine hesitancy coming to the forefront of public consciousness [81, 82]. COVID-19 vaccines were at first particularly perplexing for pregnant women, given pregnant women’s exclusion from clinical trials but increased risk for severe illness from COVID-19 . As of June 2022, 71% of US pregnant women were fully vaccinated against COVID-19, compared to 77% of adults overall . However, most (95%) vaccinated pregnant women had been fully vaccinated before becoming pregnant. Promising early data on efficacy [84,85,86] and safety [87,88,89] of mRNA COVID-19 vaccines in pregnancy were eventually published, and the American College of Obstetricians and Gynecologists (ACOG) strongly supports vaccination against COVID-19 during pregnancy .
Adaptation of MomsTalkShots for new vaccines and populations beyond English-speaking pregnant women in the US has the potential to improve vaccine knowledge and perceptions more broadly. We are currently updating and expanding MomsTalkShots to become “LetsTalkShots”, which will cover vaccines across the lifespan, including routine adolescent and adult vaccines such as HPV, influenza, and shingles. Crucially, MomsTalkShots has also been adapted to improve COVID-19 vaccine knowledge and perceptions by providing easily accessible, individually-tailored messages to assuage common concerns about COVID-19 vaccines (e.g., new technology, rushed timeline for development, long-term safety, fertility, safety in pregnancy) and appeal to populations with lower acceptance (e.g., pregnant women, ethnic minorities, younger age, less education, conservative political ideology) [91,92,93]. This new iteration of MomsTalkShots, called “LetsTalkCovidVaccines”, is free and accessible for all at http://letstalkcovidvaccines.com/. The content, design, and distribution of LetsTalkShots and LetsTalkCovidVaccines will be regularly assessed and upgraded, to reflect updates in science, incorporate new topics of concern, improve the user experience, and expand access, reach, and impact.
MomsTalkShots improved vaccine knowledge and perceptions among pregnant women and mothers. Among women initially intending not to vaccinate, MomsTalkShots increased perceived risk of maternal influenza disease and confidence in influenza vaccine efficacy; and among women with uncertain infant vaccine intentions, MomsTalkShots substantially reduced safety concerns. MomsTalkShots offers a scalable tool to address vaccine hesitancy by disseminating easily accessible information tailored to individuals’ demographics and concerns, and is currently being updated to cover vaccines across the lifespan.
Availability of data and materials
The datasets generated and analysed during our study are not publicly available due to confidentiality issues, but may be requested from the corresponding author.
- (Tdap) vaccination:
Tetanus, diphtheria, and acellular pertussis
Coronavirus disease 2019
Knowledge, attitudes and beliefs
Centers for Disease Control and Prevention
- (AFIX) model:
Assessment, Feedback, Incentives and Exchange
Continuing Medical Education
Institutional Review Board
Flu and Tdap Vaccination Coverage Among Pregnant Women – United States, 2021. https://www.cdc.gov/flu/fluvaxview/pregnant-women-apr2021.htm.
Flu Vaccination Coverage, United States, 2020–21 Influenza Season. https://www.cdc.gov/flu/fluvaxview/coverage-2021estimates.htm.
Immunization and Infectious Diseases. https://www.healthypeople.gov/2020/topics-objectives/topic/immunization-and-infectious-diseases/objectives.
Beel ER, Rench MA, Montesinos DP, Mayes B, Healy CM. Knowledge and attitudes of postpartum women toward immunization during pregnancy and the peripartum period. Hum Vaccin Immunother. 2013;9(9):1926–31.
Healy CM, Rench MA, Montesinos DP, Ng N, Swaim LS. Knowledge and attitiudes of pregnant women and their providers towards recommendations for immunization during pregnancy. Vaccine. 2015;33(41):5445–51.
Henninger ML, Irving SA, Thompson M, Avalos LA, Ball SW, Shifflett P, et al. Factors associated with seasonal influenza vaccination in pregnant women. J Women's Health (Larchmt). 2015;24(5):394–402.
MacDougall DM, Halperin BA, Langley JM, McNeil SA, MacKinnon-Cameron D, Li L, et al. Knowledge, attitudes, beliefs, and behaviors of pregnant women approached to participate in a Tdap maternal immunization randomized, controlled trial. Hum Vaccin Immunother. 2016;12(4):879–85.
Gorman JR, Brewer NT, Wang JB, Chambers CD. Theory-based predictors of influenza vaccination among pregnant women. Vaccine. 2012;31(1):213–8.
Henninger M, Naleway A, Crane B, Donahue J, Irving S. Predictors of seasonal influenza vaccination during pregnancy. Obstet Gynecol. 2013;121(4):741–9.
Payakachat N, Hadden KB, Ragland D. Promoting Tdap immunization in pregnancy: associations between maternal perceptions and vaccination rates. Vaccine. 2016;34(1):179–86.
Weiner JL, Fisher AM, Nowak GJ, Basket MM, Gellin BG. Childhood immunizations: first-time expectant mothers’ knowledge, beliefs, intentions, and behaviors. Vaccine. 2015;33(Suppl 4):D92–8.
Gellin BG, Maibach EW, Marcuse EK. Do parents understand immunizations? A national telephone survey. Pediatrics. 2000;106(5):1097–102.
Allred NJ, Shaw KM, Santibanez TA, Rickert DL, Santoli JM. Parental vaccine safety concerns: results from the National Immunization Survey, 2001-2002. Am J Prev Med. 2005;28(2):221–4.
Cacciatore MA, Nowak GJ, Evans NJ. It’s Complicated: The 2014–2015 U.S. Measles outbreak and parents’ vaccination beliefs, confidence, and intentions. Risk Analysis. 2018;38:2178–92. https://doi.org/10.1111/risa.13120.
Nowak GJ, Cacciatore MA. Parents’ confidence in recommended childhood vaccinations: extending the assessment, expanding the context. Hum Vaccin Immunother. 2017;13(3):687–700.
Chung Y, Schamel J, Fisher A, Frew PM. Influences on immunization decision-making among US parents of Young children. Matern Child Health J. 2017;21(12):2178–87.
Frew PM, Fisher AK, Basket MM, Chung Y, Schamel J, Weiner JL, et al. Changes in childhood immunization decisions in the United States: results from 2012 & 2014 National Parental Surveys. Vaccine. 2016;34(46):5689–96.
Dempsey AF, Schaffer S, Singer D, Butchart A, Davis M, Freed GL. Alternative vaccination schedule preferences among parents of young children. Pediatrics. 2011;128(5):848–56.
Freed GL, Clark SJ, Butchart AT, Singer DC, Davis MM. Parental vaccine safety concerns in 2009. Pediatrics. 2010;125(4):654–9.
Gust D, Brown C, Sheedy K, Hibbs B, Weaver D, Nowak G. Immunization attitudes and beliefs among parents: beyond a dichotomous perspective. Am J Health Behav. 2005;29(1):81–92.
Gust DA, Darling N, Kennedy A, Schwartz B. Parents with doubts about vaccines: which vaccines and reasons why. Pediatrics. 2008;122(4):718–25.
Gust DA, Strine TW, Maurice E, Smith P, Yusuf H, Wilkinson M, et al. Underimmunization among children: effects of vaccine safety concerns on immunization status. Pediatrics. 2004;114(1):e16–22.
Hofstetter AM, Simon TD, Lepere K, Ranade D, Strelitz B, Englund JA, Opel DJ. Parental vaccine hesitancy and declination of influenza vaccination among hospitalized children. Hosp Pediatr. 2018;8(10):628–35. https://doi.org/10.1542/hpeds.2018-0025.
Kennedy A, Lavail K, Nowak G, Basket M, Landry S. Confidence about vaccines in the United States: understanding parents’ perceptions. Health Aff (Millwood). 2011;30(6):1151–9.
Lavail KH, Kennedy AM. The role of attitudes about vaccine safety, efficacy, and value in explaining parents’ reported vaccination behavior. Health Educ Behav. 2013;40(5):544–51.
McCauley MM, Kennedy A, Basket M, Sheedy K. Exploring the choice to refuse or delay vaccines: a national survey of parents of 6- through 23-month-olds. Acad Pediatr. 2012;12(5):375–83.
Parrella A, Gold M, Marshall H, Braunack-Mayer A, Baghurst P. Parental perspectives of vaccine safety and experience of adverse events following immunisation. Vaccine. 2013;31(16):2067–74.
Zikmund-Fisher BJ, Wittenberg E, Lieu TA. Parental weighting of seizure risks vs. fever risks in vaccination tradeoff decisions. Vaccine. 2016;34(50):6123–5.
Smith PJ, Humiston SG, Marcuse EK, Zhao Z, Dorell CG, Howes C, et al. Parental delay or refusal of vaccine doses, childhood vaccination coverage at 24 months of age, and the health belief model. Public Health Rep. 2011;126(Suppl 2):135–46.
O’Leary ST, Brewer SE, Pyrzanowski J, Barnard J, Sevick C, Furniss A, Dempsey AF. Timing of Information-Seeking about Infant Vaccines. J Pediatr. 2018;203:125–130.e1. https://doi.org/10.1016/j.jpeds.2018.07.046.
Ellingson MK, Dudley MZ, Limaye RJ, Salmon DA, O’Leary ST, Omer SB. Enhancing uptake of influenza maternal vaccine. Expert Rev Vaccine. 2019;18(2):191–204.
Kempe A, Daley MF, McCauley MM, Crane LA, Suh CA, Kennedy AM, et al. Prevalence of parental concerns about childhood vaccines: the experience of primary care physicians. Am J Prev Med. 2011;40(5):548–55.
Leddy MA, Anderson BL, Power ML, Gall S, Gonik B, Schulkin J. Changes in and current status of obstetrician-gynecologists’ knowledge, attitudes, and practice regarding immunization. Obstet Gynecol Surv. 2009;64(12):823–9.
Patel KM, Vazquez Guillamet L, Pischel L, Ellingson MK, Bardají A, Omer SB. Strategies to increase uptake of maternal pertussis vaccination. Expert Rev Vaccine. 2021;20(7):779–96.
Limaye RJ, Opel DJ, Dempsey A, Ellingson M, Spina C, Omer SB, et al. Communicating with vaccine-hesitant parents: a narrative review. Acad Pediatr. 2021;21(4s):S24–s29.
COVID Data Tracker https://covid.cdc.gov/covid-data-tracker/#datatracker-home.
Healy J, Rodriguez-Lainz A, Elam-Evans LD, Hill HA, Reagan-Steiner S, Yankey D. Vaccination coverage among foreign-born and U.S.-born adolescents in the United States: successes and gaps - National Immunization Survey-Teen, 2012-2014. Vaccine. 2018;36(13):1743–50.
Cooper DL, Hernandez ND, Rollins L, Akintobi TH, McAllister C. HPV vaccine awareness and the association of trust in cancer information from physicians among males. Vaccine. 2017;35(20):2661–7.
Henry KA, Swiecki-Sikora AL, Stroup AM, Warner EL, Kepka D. Area-based socioeconomic factors and human papillomavirus (HPV) vaccination among teen boys in the United States. BMC Public Health. 2017;18(1):19.
Mohammed KA, Vivian E, Loux TM, Arnold LD. Factors associated with parents’ intent to vaccinate adolescents for human papillomavirus: findings from the 2014 National Immunization Survey-Teen. Prev Chronic Dis. 2017;14:E45.
Nonzee NJ, Baldwin SB, Cui Y, Singhal R. Disparities in parental human papillomavirus (HPV) vaccine awareness and uptake among adolescents. Vaccine. 2018;36(10):1243–7.
Ojeaga A, Alema-Mensah E, Rivers D, Azonobi I, Rivers B. Racial disparities in HPV-related knowledge, attitudes, and beliefs among African American and White Women in the USA. J Cancer Educ. 2019;34(1):66–72. https://doi.org/10.1007/s13187-017-1268-6.
Otanez S, Torr BM. Ethnic and racial disparities in HPV vaccination attitudes. J Immigr Minor Health. 2018;20(6):1476–82. https://doi.org/10.1007/s10903-017-0685-2.
Quinn SC, Jamison A, Freimuth VS, An J, Hancock GR, Musa D. Exploring racial influences on flu vaccine attitudes and behavior: results of a national survey of white and African American adults. Vaccine. 2017;35(8):1167–74.
Warner EL, Ding Q, Pappas LM, Henry K, Kepka D. White, affluent, educated parents are least likely to choose HPV vaccination for their children: a cross-sectional study of the National Immunization Study - teen. BMC Pediatr. 2017;17(1):200.
Dudley MZ, Limaye RJ, Salmon DA, Omer SB, O’Leary ST, Ellingson MK, Spina CI, Brewer SE, Bednarczyk RA, Malik F, Frew PM, Chamberlain AT. Racial/ethnic disparities in maternal vaccine knowledge, attitudes, and intentions. Public Health Rep. 2021;136(6):699–709. https://doi.org/10.1177/0033354920974660.
Bednarczyk RA, Chamberlain A, Mathewson K, Salmon DA, Omer SB. Practice-, provider-, and patient-level interventions to improve preventive care: development of the P3 model. Prev Med Rep. 2018;11:131–8.
Omer SB, O’Leary ST, Bednarczyk RA, Ellingson MK, Spina CI, Dudley MZ, et al. Multi-tiered intervention to increase maternal immunization coverage: a randomized, controlled trial. Vaccine. 2022;40(34):4955–63.
Salmon DA, Limaye RJ, Dudley MZ, Oloko OK, Church-Balin C, Ellingson MK, et al. MomsTalkShots: An individually tailored educational application for maternal and infant vaccines. Vaccine. 2019;37(43):6478–85.
Dudley MZ, Salmon DA, Halsey NA, Orenstein WA, Limaye RJ, O’Leary ST, et al. The Clinician’s vaccine safety resource guide: optimizing prevention of vaccine-preventable diseases across the lifespan. Cham: Springer International Publishing; 2018. https://link.springer.com/book/10.1007/978-3-319-94694-8.
Frew PM, Randall LA, Malik F, Limaye RJ, Wilson A, O'Leary ST, et al. Clinician perspectives on strategies to improve patient maternal immunization acceptability in obstetrics and gynecology practice settings. Hum Vaccin Immunother. 2018;14(7):1548–57.
Dempsey AF, Pyrznawoski J, Lockhart S, Barnard J, Campagna EJ, Garrett K, et al. Effect of a health care professional communication training intervention on adolescent human papillomavirus vaccination: a cluster randomized clinical trial. JAMA Pediatr. 2018;172(5):e180016.
Opel DJ, Robinson JD, Spielvogle H, Spina C, Garrett K, Dempsey AF, et al. ‘Presumptively initiating vaccines and optimizing talk with motivational Interviewing’ (PIVOT with MI) trial: a protocol for a cluster randomised controlled trial of a clinician vaccine communication intervention. BMJ Open. 2020;10(8):e039299.
Brewer NT, Hall ME, Malo TL, Gilkey MB, Quinn B, Lathren C. Announcements versus conversations to improve HPV vaccination coverage: a randomized trial. Pediatrics. 2017;139(1):e20161764. https://doi.org/10.1542/peds.2016-1764.
Dudley MZ, Limaye RJ, Omer SB, O'Leary ST, Ellingson MK, Spina CI, et al. Factors associated with referring close contacts to an app with individually-tailored vaccine information. Vaccine. 2020;38(13):2827–32.
Dudley MZ, Taitel MS, Smith-Ray R, Singh T, Limaye RJ, Salmon DA. Effect of educational and financial incentive-based interventions on immunization attitudes, beliefs, intentions and receipt among close contacts of pregnant women. Vaccine. 2021;39(6):961–7.
Spina CI, Brewer SE, Ellingson MK, Chamberlain AT, Limaye RJ, Orenstein WA, et al. Adapting Center for Disease Control and Prevention's immunization quality improvement program to improve maternal vaccination uptake in obstetrics. Vaccine. 2020;38(50):7963–9.
Chamberlain AT, Limaye RJ, O'Leary ST, Frew PM, Brewer SE, Spina CI, et al. Development and acceptability of a video-based vaccine promotion tutorial for obstetric care providers. Vaccine. 2019;37(19):2532–6.
Chamberlain AT, Seib K, Ault KA, Rosenberg ES, Frew PM, Cortes M, et al. Improving influenza and Tdap vaccination during pregnancy: a cluster-randomized trial of a multi-component antenatal vaccine promotion package in late influenza season. Vaccine. 2015;33(30):3571–9.
O'Leary ST, Pyrzanowski J, Brewer SE, Dickinson LM, Dempsey AF. Evidence-based vaccination strategies in obstetrics and gynecology settings: current practices and methods for assessment. Hum Vacc Immunother. 2016;12(4):866–71.
Centers for Disease C, Prevention. Updated recommendations for use of tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis vaccine (Tdap) in pregnant women--advisory committee on immunization practices (ACIP), 2012. MMWR Morb Mortal Wkly Report. 2013;62(7):131–5.
Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215.
Cialdini RB. Crafting normative messages to protect the environment. Curr Dir Psychol Sci. 2003;12(4):105–9.
Opel DJ, Mangione-Smith R, Taylor JA, Korfiatis C, Wiese C, Catz S, et al. Development of a survey to identify vaccine-hesitant parents: the parent attitudes about childhood vaccines survey. Hum Vaccin. 2011;7(4):419–25.
Dudley MZ, Limaye RJ, Omer SB, O'Leary ST, Ellingson MK, Spina CI, et al. Characterizing the vaccine knowledge, attitudes, beliefs, and intentions of pregnant women in Georgia and Colorado. Hum Vaccin Immunother. 2020;16(5):1109–17.
Increasing Appropriate Vaccination. https://www.thecommunityguide.org/topic/vaccination.
Meharry PM, Cusson RM, Stiller R, Vazquez M. Maternal influenza vaccination: evaluation of a patient-centered pamphlet designed to increase uptake in pregnancy. Matern Child Health J. 2014;18(5):1205–14.
Williams SE, Rothman RL, Offit PA, Schaffner W, Sullivan M, Edwards KM. A randomized trial to increase acceptance of childhood vaccines by vaccine-hesitant parents: a pilot study. Acad Pediatr. 2013;13(5):475–80.
Vannice KS, Salmon DA, Shui I, Omer SB, Kissner J, Edwards KM, et al. Attitudes and beliefs of parents concerned about vaccines: impact of timing of immunization information. Pediatrics. 2011;127(Suppl 1):S120–6.
Glanz JM, Wagner NM, Narwaney KJ, Kraus CR, Shoup JA, Xu S, O’Leary ST, Omer SB, Gleason KS, Daley MF. Web-based social media intervention to increase vaccine acceptance: a randomized controlled trial. Pediatrics. 2017;140(6):e20171117. https://doi.org/10.1542/peds.2017-1117.
O'Leary ST, Narwaney KJ, Wagner NM, Kraus CR, Omer SB, Glanz JM. Efficacy of a web-based intervention to increase uptake of maternal vaccines: An RCT. Am J Prev Med. 2019;57(4):e125–33.
Fisher JS, Radvansky GA. Patterns of forgetting. J Mem Lang. 2018;102:130–41.
Pluviano S, Watt C, Della Sala S. Misinformation lingers in memory: failure of three pro-vaccination strategies. PLoS One. 2017;12(7):e0181640.
Nyhan B, Reifler J, Richey S, Freed GL. Effective messages in vaccine promotion: a randomized trial. Pediatrics. 2014;133(4):e835–42.
Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–64.
Dempsey AF, Maertens J, Beaty B, O’Leary ST. Characteristics of users of a tailored, interactive website for parents and its impact on adolescent vaccination attitudes and uptake. BMC Res Notes. 2015;8:739.
Atkinson KM, Westeinde J, Ducharme R, Wilson SE, Deeks SL, Crowcroft N, et al. Can mobile technologies improve on-time vaccination? A study piloting maternal use of ImmunizeCA, a Pan-Canadian immunization app. Hum Vaccin Immunother. 2016;12(10):2654–61.
Fadda M, Galimberti E, Fiordelli M, Romano L, Zanetti A, Schulz PJ. Effectiveness of a smartphone app to increase parents’ knowledge and empowerment in the MMR vaccination decision: a randomized controlled trial. Hum Vaccin Immunother. 2017;13(11):2512–21.
Bednarczyk RA, Frew PM, Salmon DA, Whitney E, Omer SB. ReadyVax: a new mobile vaccine information app. Hum Vaccin Immunother. 2017;13(5):1149–54.
Real FJ, Rosen BL, Bishop JM, McDonald S, DeBlasio D, Kreps GL, Klein M, Kahn JA. Usability evaluation of the novel smartphone application, HPV vaccine: same way, same day. Among Pediatric Residents Acad Pediatr. 2021;21(4):742–9. https://doi.org/10.1016/j.acap.2020.11.023.
Salmon D, Opel DJ, Dudley MZ, Brewer J, Breiman R. Reflections on governance, communication, and equity: challenges and opportunities in COVID-19 vaccination. Health Aff (Millwood). 2021;40(3):419–25.
Dudley MZ, Bernier R, Brewer J, Salmon DA. Walking the tightrope: reevaluating science communication in the era of COVID-19 vaccines. Vaccine. 2021;39(39):5453–5.
Pregnant and Recently Pregnant People At Increased Risk for Severe Illness from COVID-19. https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/pregnant-people.html.
Gray KJ, Bordt EA, Atyeo C, Deriso E, Akinwunmi B, Young N, et al. Coronavirus disease 2019 vaccine response in pregnant and lactating women: a cohort study. Am J Obstet Gynecol. 2021;225(3):303.e301–17.
Dagan N, Barda N, Biron-Shental T, Makov-Assif M, Key C, Kohane IS, et al. Effectiveness of the BNT162b2 mRNA COVID-19 vaccine in pregnancy. Nat Med. 2021;27(10):1693–5.
Goldshtein I, Nevo D, Steinberg DM, Rotem RS, Gorfine M, Chodick G, et al. Association between BNT162b2 vaccination and incidence of SARS-CoV-2 infection in pregnant women. JAMA. 2021;326(8):728–35.
Bowman CJ, Bouressam M, Campion SN, Cappon GD, Catlin NR, Cutler MW, et al. Lack of effects on female fertility and prenatal and postnatal offspring development in rats with BNT162b2, a mRNA-based COVID-19 vaccine. Reprod Toxicol. 2021;103:28–35.
Shimabukuro TT, Kim SY, Myers TR, Moro PL, Oduyebo T, Panagiotakopoulos L, et al. Preliminary findings of mRNA Covid-19 vaccine safety in pregnant persons. N Engl J Med. 2021;384(24):2273–82.
Kharbanda EO, Haapala J, DeSilva M, Vazquez-Benitez G, Vesco KK, Naleway AL, et al. Spontaneous abortion following COVID-19 vaccination during pregnancy. JAMA. 2021;326(16):1629–31.
COVID-19 Vaccination Considerations for Obstetric–Gynecologic Care. https://www.acog.org/clinical/clinical-guidance/practice-advisory/articles/2020/12/covid-19-vaccination-considerations-for-obstetric-gynecologic-care.
Growing Share of Americans Say They Plan To Get a COVID-19 Vaccine – or Already Have. https://www.pewresearch.org/science/2021/03/05/growing-share-of-americans-say-they-plan-to-get-a-covid-19-vaccine-or-already-have/.
KFF COVID-19 Vaccine Monitor: 2021 https://www.kff.org/coronavirus-covid-19/poll-finding/kff-covid-19-vaccine-monitor-february-2021/.
Two-Thirds of Americans Not Satisfied With Vaccine Rollout. https://news.gallup.com/poll/329552/two-thirds-americans-not-satisfied-vaccine-rollout.aspx.
We appreciate Tina Proveaux (Johns Hopkins Institute for Vaccine Safety) for her help in the final formatting and submission of this manuscript.
This work was supported by the National Institute for Allergy and Infectious Diseases at the National Institutes of Health [grant number R01AI110482]. The funder had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. ClinicalTrials. Gov registration number: NCT02898688.
Ethics approval and consent to participate
All methods were carried out in accordance with relevant guidelines and regulations. Informed consent was obtained from all participants. The study was approved by the Institutional Review Board (IRB) of Emory University and Sisters of Charity of Leavenworth Health System. Johns Hopkins University and University of Colorado Denver ceded approval to the Emory University IRB.
Consent for publication
RJL, SBO, STO, MKE, CIS, SEB, RAB, FM, and CCB declare that they have no competing interests. MZD has received some research support from Merck and Walgreens. ATC received paid consultancy with the American College of Obstetricians and Gynecologists regarding provider-to-patient communications. During this study, PMF had no conflicts or financial disclosures to report; at the time of publication, she is at Merck. WAO is a member of the Scientific Advisory Board for Moderna. KAA is a current member of the federal Advisory Committee on Immunization Practices and is also a member of a similar advisory committee for the American College of Obstetricians and Gynecologists. LER receives consultant fees from GlaxoSmithKline, writer fees from Up to Date and compensation for being on the editorial boards of Contemporary OB/GYN and the New England Journal of Medicine. NAH has received consultant fees from Takeda, Merck and Inovio for service on Data and Safety Monitoring Boards, and from Pfizer and Valneva for service on advisory panels. DAS has received consulting and/or research support form Janssen, Merck, Walgreens, and Pfizer.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Impact of MomsTalkShots on Vaccine KABs of Women One Month After Their Infant's Birth individual survey items, organized by constructs, stratified by baseline vaccine intentionsa, dichotomous analysisb. Appendix 2. Impact of MomsTalkShots on Vaccine KABs of Women One Year After Their Infant's Birth individual survey items, organized by constructs, stratified by baseline vaccine intentionsa, dichotomous analysisb. Appendix 3. Impact of MomsTalkShots on Vaccine KABs of Women One Month After Their Infant's Birth individual survey items, organized by constructs, stratified by baseline vaccine intentionsa, continuous analysisb. Appendix 4. Impact of MomsTalkShots on Vaccine KABs of Women One Year After Their Infant's Birth individual survey items, organized by constructs, stratified by baseline vaccine intentionsa, continuous analysisb. Appendix 5. Impact of MomsTalkShots on Women's Perceived Risk of Infant Pertussis One Month After Their Infant's Birth individual survey items, stratified by baseline vaccine intentionsa and Tdap vaccination, dichotomous analysisb. Appendix 6. Impact of MomsTalkShots on Women's Perceived Risk of Infant Pertussis One Year After Their Infant's Birth individual survey items, stratified by baseline vaccine intentionsa and Tdap vaccination, dichotomous analysisb. Appendix 7. Survey Initiation and Completion stratified by study arm. Appendix 8. Follow-Up Survey Completion stratified by maternal and infant vaccine intentions at baseline. Appendix 9. Response Rates for Vaccine KAB Constructs at Each Study Timepoint. Appendix 10. Maternal and Infant Vaccine Intentions at Baseline stratified by 4 study arms.
About this article
Cite this article
Dudley, M.Z., Omer, S.B., O’Leary, S.T. et al. MomsTalkShots, tailored educational app, improves vaccine attitudes: a randomized controlled trial. BMC Public Health 22, 2134 (2022). https://doi.org/10.1186/s12889-022-14498-7