The evolution from simple reminders and cues to action to more in-depth behavior change programs raises the importance of behavior change theory for mHealth. Recent studies have noted both that the relevance of existing behavioral theory, such as Social Cognitive Theory (SCT) and Theory of Planned Behavior (TPB), need to be examined and new models considered for their applicability to mHealth [5, 11, 13–15]. There is also a need for evaluation studies based on behavior change theory, as previous mHealth evaluations have not tested the theory-based mechanisms (mediators) of behavior change that have been shown to predict adoption and maintenance of health behaviors in previous research [13]. Evaluation models based on theory need to be developed and tested across multiple subject areas.
Text4baby is a text messaging service launched in February 2010 that delivers text messages (http://www.text4baby.org) to pregnant women and new mothers. Text4baby specifically targets traditionally underserved women facing health disparities. The intervention sends texts messages to offer immediate, “just-in-time” tips, with the goal of improving prenatal and postpartum health outcomes [13]. Text4baby aims to increase maternal expectations for health outcomes and to promote self-efficacy to utilize the health care system and make informed health care choices, outcomes that may not be regularly achieved in a population facing health disparities and with limited health care access. The program is based on traditional behavioral theories including Social Cognitive Theory (SCT), the Transtheoretical Model (TTM), and the Health Belief Model (HBM) [14, 16, 17]. Following elements of these theories, text4baby seeks to build self-efficacy to successfully utilize health care, improve health literacy, and increase expectations for successful pregnancy and new motherhood. It is designed to build knowledge and skills to manage one’s own health and prevent health risks by avoiding smoking, drinking, receiving recommended immunizations, and avoiding similar behavioral risk factors [18].
Figure 1 presents a preliminary model of text4baby that has been used to guide the development of program evaluation strategies.
We evaluated pre-natal text messages (new baby messages were not part of this study) delivered by the text4baby program in a randomized pilot study at two clinics that are part of the Fairfax County, Virginia Health Department. Women who initially presented for pre-natal care (first visit) at the Fairfax County, Virginia Health Department and then participate in the InovaCares clinic in Fairfax County were eligible for the study. A majority of the client population spoke Spanish as their primary language. Those who agreed to participate and provide verbal informed consent were randomly assigned into a text4baby exposure or to a no exposure (control) group. Both groups received standard Inova pre-natal counseling and care during study participation. Participants in the text4baby exposure group were directed to enroll in the text4baby message service by clinic staff implementing the evaluation study, in order to receive messages for the duration of their pregnancy (or until they dropped out of the program). No incentives were provided for participation in the study. Among womwn who presented at the clinics for prenatal services, 147 agreed to enroll in the study, 123 women responded to the baseline telephone survey (83.7% retention rate). Among the 123 enrollees, 90 completed a follow-up survey, resulting in a 73% retention rate.
Design and measures
Respondents enrolled in the study completed a 24-item interviewer administered questionnaire developed by the principal investigator. Baseline data collection started in April 2011 and ended in January 2012. Follow-up data collection was completed in April 2012. The survey instrument contained a battery of questions on participant attitudes and behaviors concerning nutrition, smoking and health information-seeking; demographic information such as age, race, ethnicity, education, zip code, marital status and primary language was collected from Fairfax County, Virginia Health Department. The instrument was pre-tested with 7 Spanish-speaking respondents similar to the target population who were debriefed about item comprehension and not included in the subsequent evaluation. The final instrument incorporated revisions based on pretesting. The study protocol was approved as minimal risk research by the George Washington University and Fairfax County, Virginia Health Department institutional review boards on March 15, 2011 (GWU approval number 111047).
The variables for behavioral outcomes were derived from existing, validated instruments, including the Behavioral Risk Factor Surveillance Survey (BRFSS) and National Health and Nutritional Examination Survey (NHANES). An example behavioral outcome variable includes the following: “During the last 3 months, about how many servings of fruit did you have in a day?,” with the following response options [zero servings, 1 or 2 servings per day, 3 or 4 servings per day, 5 or more servings per day or Don’t Know]. Variables specific to attitudes and beliefs were adapted from these same sources and validated instruments used by the investigators in previous research [19, 20]. Example attitudes and beliefs variables include the following: “Eating 5 or more fruits and vegetables per day is important to the health of my developing baby,” and “Taking a prenatal vitamin is important to the health of my developing baby,” with the following response options [Strongly Agree, Agree, Disagree, Strongly Disagree, Don’t Know]. Variables specific to the text messages delivered by the text4baby mHealth program such as confirmed recall and reactions and receptivity to the messages were adapted by the authors based on validated measures previously published in social marketing evaluation research, including their own work [21, 22].
Prior to telephone survey implementation, each new participant was recruited into the evaluation study. The clinical intake staff (mainly nurses), with the assistance of Spanish language translators when necessary, conducted all recruitment activities in-person at the Fairfax County, Virginia Health Department clinic sites. Clinical intake staff requested and drafted a script in conjunction with the research staff to use as a tool to assist in introducing the study, and in following protocol procedures that instructed participants, assigned to the treatment condition, to enroll in the text4baby message service after consent. Those who agreed to participate and provide verbal informed consent detailing study procedures were randomly assigned into a text4baby exposure or control group based on a randomly generated list of the treatment and control conditions. Clinical intake staff used a pre-generated, randomly ordered list of group assignments in this process. Participants in the text4baby exposure group received standard InovaCares pre-natal counseling and care in addition to the text4baby messages. Control participants received only standard InovaCares pre-natal counseling and care. No further recruitment efforts were made beyond the individual's visit to the Health Department clinic sites. Individuals who received services but did not consent to the study received no further contact for recruitment.
Participants in the text4baby exposure group were instructed to enroll in the text4baby message service to receive messages for the duration of their pregnancy (or until they dropped out of the program). These participants enrolled by texting 'BABY' to the short code 511411 (standard text4baby enrollment procedure). In addition, they texted the keyword 'CARES' after 'BABY' to signify that they enrolled in the Inova study population (meant to identify them as participants in this study). During implementation, we added a reminder card that stated these instructions to enroll in the text4baby service (in English and Spanish), for participants to be able to review the instructions provided by the clinic staff. Upon successful recruitment and enrollment into the text4baby message service (participants agreed to study involvement and completed informed consent), their pregnancy due date, zip code, mobile phone number, and the CARES keyword was uploaded to a database managed by Voxiva, Inc. Voxiva is the information technology firm that delivers the automated text4baby messages to enrollees. We interviewed participants about their knowledge, attitudes, beliefs, and behaviors concerning pre-natal care, nutrition, physical activity, substance use, vitamins, immunizations, and related health behaviors and risk factors, the primary outcome variables of interest for this study. Participants were followed up, in the same manner, approximately 2–3 months after their initial pre-natal visit (meant to coincide with the participants’ follow-up visit at the InovaCares clinic later in their pregnancy). The purpose of the follow-up interview was to identify differences between the exposure and control groups potentially due to text4baby exposure. For text4baby exposure group participants, the follow-up interview gathered data about their exposure and reactions to the text4baby campaign and its messages in order to determine the effectiveness of text4baby messages on maternal, pre-natal care and related health knowledge, attitudes, beliefs, and behavioral outcomes, including attending pre-natal care visits, nutrition, taking vitamins, getting flu shots, avoiding smoking and related health promoting and risk avoidance behaviors.
Sampling
We randomly sampled from a largely low-income population of women initially seeking pre-natal care from the Fairfax Health Department (who then participated in InovaCares clinic services). The target sample size for the study was 260 participants. This number was estimated based on two factors. First, we estimated a required sample of 130 per comparison group (text4baby treatment plus usual care compared to usual care alone) to estimate a 15% difference in repeat (more than one visit) pre-natal care utilization (75% for usual care versus 90% in the text4baby exposed group), with a design effect of 1.5. Second, based on enrollment figures for the early months of 2010 (1 year before proposed data collection, same time of year) approximately 500 women presented for care at the same clinics, indicating a sufficient sample from which to recruit during the planned 2 month recruitment period (later extended to approximately 10 months). Recruitment proved difficult for clinical staff operating in their natural setting, preventing us from reaching the target sample size of 260, however, the resulting sample size was adequate for multivariable logistic analyses as confirmed by the consulting statistician on the project. These issues were limitations to the study, as discussed below.
Data collection procedures
Before data collection initiation, we held an introductory meeting and training sessions at both Fairfax County, Virginia Health Department clinics describing the purpose and importance of the study, study procedures and protocol for clinic staff. Individuals approached and agreeing to enroll in the study were contacted by a George Washington University (GWU) interviewer, supervised by the Principal Investigator (PI), by telephone to complete a pre-exposure, baseline questionnaire. The GWU interviewer was a trained research assistant, fluent in English and Spanish. The telephone-administered survey was approximately 15 min in length; control group participants were also administered the same baseline survey. The GWU interviewer checked faxes received by the 2 clinics on a daily basis during the recruitment period and immediately conducted baseline interviews with newly enrolled participants. Baseline and follow-up interviews were conducted by phone to the participants’ mobile phone (or land-line in cases when participants explicitly requested to carry out the interview on a land-line phone). Enrolled participant names were provided by the clinics and were used solely to develop communication during the phone interview; names were not recorded in the survey database to help execute confidentiality safeguards outlined in the study protocol. At follow-up, we used the same behavioral survey instrument with the addition of a short battery of questions about exposure and reactions to the T4B campaign and its messages.
Data analysis
We used multilevel logistic regression to construct separate models for each of the attitudes, beliefs, and behavioral outcomes (i.e. fruit and vegetable consumption). Utilizing an interaction term, we estimated the odds of positive change over time in response to each of the behavioral outcome variables as a function of text4baby text message exposure, and also as a function of educational status (dichotomized as less than high-school versus high-school or greater). Education status is an important socio-demographic variable that has been shown to moderate effects of health communication interventions such as the one tested in this study and was included as an effect modifier a priori [22, 23]. We modeled these outcomes using a generalized estimating equation (GEE) with a multivariable logistic regression model, which allows us to assess population-averaged effects of our predictors, while accounting for correlation of responses within individuals from baseline to follow-up. Participant, age and marital status were included as covariates in each model. Robust sandwich estimators were used to compute standard errors, and an independence working model for correlation was utilized for the generalized estimating equation [24]. Stata Version IC 12.1 (College Station, Texas) was used for the analysis.
The GEE model specification may be expressed in the following formula
(1)
where i = between-subject value, j = within-subject, between-measurement value, time = indicator of pre- or post- measurement value for each subject (1/0), interv = exposure to text-4-baby intervention (1/0), educ = high-school or greater education (1/0), agel20 = maternal age of < 20 years at pre-intervention measurement (1/0), ageg35 = maternal age of > 35 years at pre-intervention measurement (1/0), marital = marital status at pre-intervention measurement (1/0).
For missing data, we excluded cases where we lacked complete outcome data for analysis. We ran a t-test to compare covariates, including socio-demographic and other variables used in our regressions, between cases with and without missing data to verify whether or not data were missing completely at random. Non-response dependent on covariates was adjusted for in the models.