- Research article
- Open Access
A gloomy picture: a meta-analysis of randomized controlled trials reveals disappointing effectiveness of programs aiming at preventing child maltreatment
BMC Public Health volume 15, Article number: 1068 (2015)
Consistent findings about the effectiveness of parent programs to prevent or reduce child maltreatment are lacking.
In the present meta-analysis we synthesized findings from 27 independent samples from randomized controlled trials (RCTs) on the effectiveness of 20 different intervention programs aimed at (i) preventing the occurrence of child maltreatment in the general population or with at-risk but non-maltreating families, or (ii) reducing the incidence of child maltreatment in maltreating families.
A significant combined effect on maltreatment (d = 0.13; N = 4883) disappeared after the trim-and-fill approach that takes into account publication bias against smaller studies without significant outcomes. However, moderator analyses showed that larger effect sizes were found for more recent studies, studies with smaller samples, programs that provide parent training instead of only support, programs that target maltreating instead of at-risk families, and programs with a moderate length (6–12 months) or a moderate number of sessions (16–30).
More RCTs are needed to further unravel which factors are associated with program effectiveness. Because currently existing programs appeared to only reduce and not prevent child maltreatment, efforts in the field of preventive intervention should also focus on the development and testing of preventive programs for families at risk for child maltreatment.
The number of parent support programs aimed at preventing or reducing child maltreatment has grown over the last decades. Some of these programs were found to have a positive impact on various parenting domains in studies using randomized controlled designs (RCTs; ). However, consistent findings about the effectiveness of such programs to prevent or reduce child maltreatment are lacking [2, 3]. The current meta-analysis aims to fill this gap. We synthesized findings of all randomized controlled trials (23 studies) that tested the effectiveness of 20 different programs, aimed at the general population, at-risk, and maltreating groups, in order to reveal the overall success of programs to prevent or reduce the occurrence of child maltreatment and to uncover factors that influence the effectiveness of intervention programs.
A recent series of meta-analyses indicated that child maltreatment is a serious problem, affecting children all over the world. Worldwide prevalence rates of different types of maltreatment ranged from 0.3 % based on studies with reports from professionals to 36.3 % based on self-report studies . Risk factors for child maltreatment are low socio-economic status, parental mental health problems, family isolation, and single parenthood [5–7]. Child maltreatment is associated with short-term and long-term negative consequences. Victims have an increased risk for physical, behavioral, and psychological problems, also up into adulthood (e.g., [8–11]), and benefit less from treatment compared to non-maltreated individuals , leading to high costs for individuals and society. Given the high prevalence rates and serious consequences of maltreatment, effective prevention and reduction of child maltreatment is essential.
Over the last decades, the number of parent support programs has increased exponentially . Most of these programs are targeted and provide support to a clearly defined population identified on the basis of risk factors for child maltreatment. However, some programs are available for everyone or at least for a large proportion of the population. Examples of such universal programs are Triple-P  and SOS! Help for Parents . These programs aim to prevent the occurrence of child maltreatment in the general population, for example by using the media to inform parents about effective parenting strategies or by providing a short parent skill training to parents who visit a well-baby clinic. Concerning programs that target a clearly defined population, programs that prevent the occurrence of child maltreatment in at-risk, but non-maltreating families, can be distinguished from programs that reduce the incidence of child maltreatment in maltreating families.
A well-known targeted prevention program is the Nurse-Family Partnership developed by Olds and colleagues (e.g., [15, 16]). This program specifically targets pregnant adolescent women who are unmarried and/or have a low income, but women without any of these risk factors are also allowed to participate in the program. It consists of nurse home visits in the prenatal period and during the first two years of the child's life. The nurses promote improvement of the women's health behavior during and after pregnancy, help building supportive relationships with family and friends, and link them with other needed services. The Elmira (New York) trial indicated a significant difference of 80 % fewer child maltreatment cases in the intervention group compared to the control group during the period of intervention. However, these positive results disappeared in the two years after the end of the program .
Parent–child Interaction Therapy (PCIT) is an example of a targeted program that aims to reduce the incidence of child maltreatment in physically abusive parents. Families receive 14 weekly one-hour live-coached sessions of parent–child interaction training. The training consists of child-directed interaction, in which the parent is instructed to follow the child's lead, and parent-directed interaction in which the parent is taught to direct the child's behavior and use consistent disciplinary techniques . Several studies have shown that PCIT indeed effectively reduces child behavior problems [18, 19], and an RCT also indicated significantly fewer reports of physical abuse and improved parenting skills in the PCIT condition compared to families who received community services .
Prior meta-analytic findings
A number of meta-analyses have synthesized results on the effectiveness of intervention programs aimed at preventing or reducing child maltreatment. However, some meta-analyses did not specifically include papers that measured the actual occurrence of child maltreatment [21, 22], focused solely on non-maltreating families [23–25], included only home-visiting programs [23, 25, 26], and/or included studies with less rigorous designs than RCTs [21, 23, 24]. For instance, Layzer and colleagues  combined abuse and neglect outcomes with child injuries, accidents, and removal from the home into a single category 'child safety', which makes it impossible to estimate the actual ability of programs to prevent or reduce child maltreatment. Geeraert and colleagues  examined the effect of early prevention programs on actual abuse and neglect, but they included mostly nonrandomized designs. A significant but small overall effect on reported child maltreatment was found, but moderator analyses were not conducted. Similarly, Filene and colleagues  examined the effect of home visiting programs on child maltreatment, but they also included nonrandomized designs, and did not include maltreating families, thereby only examining the preventive effect of interventions. In contrast to Geeraert and colleagues, these authors did not find a significant effect on child maltreatment. In another meta-analysis, only RCTs were included, but the focus of this meta-analysis was solely on programs starting during pregnancy or within 6 months after birth . It revealed a small but significant effect for maltreatment outcomes at the end of intervention, but no effect at follow-up. The only significant moderator that was identified for child abuse and neglect measures was year of publication; more recent studies yielded smaller effect sizes.
The current study: Program effectiveness and moderators
The current meta-analysis aims to estimate the average effect of intervention programs that provide services to parents in order to prevent or reduce child maltreatment. We only included RCTs, in which participants are fully randomly assigned to either the intervention or the control condition. Because of the random assignment, it can be assumed that the two groups do not differ systematically before the start of the program. Clustered randomized trials were excluded, because participants are not fully randomly assigned and therefore participants (or their contexts) in one cluster may not be comparable to participants in other clusters. Further, we aimed to include three types of programs: those targeting the general population, aimed at preventing maltreatment, those for families at risk for child maltreatment, aimed at preventing maltreatment, and those specifically developed for maltreating families, aimed at reducing maltreatment. We only included studies if they reported on actual maltreatment outcomes and used this outcome in our meta-analysis. Child maltreatment was defined as “any act or series of acts of commission or omission by a parent or other caregiver that results in harm, potential for harm, or threat of harm to a child” (Centers for Disease Control and Prevention (CDC)). In addition, we examined whether various intervention, design, sample, and study characteristics were associated with program effects.
An important characteristic of the intervention is the focus of the program. In some programs, parents receive various sorts of support (e.g., social, emotional, material) in order to build on strengths and improve overall family functioning, without actual parenting skills training. For example, in Healthy Families America parents receive support to reduce social isolation, access recourses such as food, housing, employment, and health care, and improve their knowledge about child development . Other programs do provide actual training for parents to improve their parenting skills, such as SOS! Help for parents , in which parents are instructed about (the role of) parenting skills and common mistakes in parenting, or Parent Child Interaction Therapy , in which parents receive (among other things) live parent–child coaching sessions to improve parent–child interaction skills. Finally, some intervention programs combine parent training and support. For example, in the Project Support intervention , mothers are taught skills for child behavioral management by instruction, practice, and feedback, and they are provided with instrumental and emotional support, such as training in how to evaluate a child care provider.
Further, the way of delivery is another intervention characteristic that can differ substantially between programs. Some programs use support-groups in a center-based setting , others consist of personal home visits  or combine center-based and home-based sessions . The number of sessions and the duration varies from program to program. For instance, in the Nurse-Family Partnership Program , parents receive 45 home visits during the first two years of the their child's life, while the SOS! Help for parents program (SOS) described by Oveisi and colleagues  consists of only two 2-h weekly sessions. A meta-analysis on the effectiveness of interventions aimed at improving parental sensitivity and parent–child attachment revealed that programs with fewer contacts were more effective in improving sensitivity and attachment , but it is unclear if this is also true for programs aimed at preventing or reducing child maltreatment. Last, and more specific for programs aimed at preventing child maltreatment, the moment of onset of the program, and thus the age of the child at the start of the program, has been discussed as an important moderator of a program's effectiveness. Although it has been suggested that programs for the prevention of child maltreatment would be most effective if starting before birth , meta-analytic evidence showed that programs focusing on parental sensitivity or parent–child attachment that started 6 months after birth were at least as effective as programs with an earlier onset .
Intervention programs target different populations. Universal programs target the general population, while targeted programs focus on a clearly defined group of families at risk for child maltreatment or maltreating families. Some have suggested that programs with a clear target population would be more effective . This may be especially true for programs that target maltreating families, because those families show the behaviors that are targeted for change, and therefore they may have the greatest potential for demonstrating change.
The rigor of the study design may also affect the effect size. Studies with poorer methodological designs likely yield larger effect sizes . The use of intent-to-treat analyses is an example of a methodological strength, as selective refusals after randomization or selective attrition during the intervention may affect the randomization. In intent-to-treat analyses, group differences are analyzed based on the original random assignment. Other design characteristics are sample size, whether assessment was blind for group assignment and whether a pretest was included. Moreover, the type and amount of services received in the control condition differs between programs. Largest effect sizes may be expected when the control group received few or even no services. In addition, there may be differences in effect sizes for short-term and long-term effects. On the one hand, it may be expected that intervention effects decrease or even disappear over time. On the other hand, there may be sleeper effects, meaning that intervention effects increase over time, because parents would need some more time to practice new skills . Finally, the method of assessment of child maltreatment may influence effect sizes. Although self-report measures may be informative since participants may know their own experiences best, self-reports have several disadvantages. Participants may interpret definitions of maltreatment or parenting practices differently than researchers and it may be difficult for participants to remember the exact frequency of specific events in the past. In addition, self-report of maltreatment experience is not possible in early childhood. In contrast, reports from professionals who work with children do cover all ages and these reports are generally coded by expert coders who use the same set of definitions. The downside of this method is that professionals may not be aware of all cases of maltreatment; they may only see the tip-of-the-iceberg .
Eligible studies were identified using a systematic search in three electronic databases (Web of Science, ERIC, and PsychInfo) using the terms child abuse and/or child neglect and/or child maltreatment combined with interven* or preven*, and written in English. Studies published up until the end of 2012 were identified, and no earliest time point was specified. Articles were included if they described (1) empirical studies in which (2) a randomized controlled trial (RCT) was applied to test the effectiveness of (3) an intervention program for the general population, families at risk for maltreatment or maltreating families with child maltreatment as outcome measure (4) in non-clinical samples. The Child Abuse Potential Inventory (CAPI) was not included as a measure of maltreatment (e.g., 35]). Articles were excluded if not enough information was provided for calculation of effect sizes. For instance, the number of families that received the Triple P intervention in the population based trial described by Prinz and colleagues  is unclear, making it impossible to calculate an effect size. Another reason for excluding this Triple P trial was the use of clustered randomization. If publications reported on the same sample or on overlapping samples, the publication providing the maximum of information was included in the meta-analysis. Thus, the independence of samples and the inclusion of every participant only once in the meta-analysis were ascertained. When possible and necessary, the coding form for the study was supplemented with information from the excluded publication on the same sample.
The initial search procedure identified 5108 articles. First, 651 duplicate articles were excluded. After screening the abstracts of the remaining articles, another 4083 articles were excluded, because they did not meet inclusion criteria. From the 374 full-text articles assessed for eligibility, 349 articles were excluded from the meta-analysis because they did not meet the inclusion criteria or were non-retrievable (see Fig. 1 for a flow chart and Additional file 1 for the PRISMA checklist). The final sample of 23 eligible articles covered reports on 27 independent samples with maltreatment outcomes in the general population, at-risk and maltreating groups (N = 4883 families; see Table 1).
Because of the strict inclusion criteria, some effectiveness studies testing well-known interventions programs were not included in this meta-analysis. For instance, for the Nurse-Family Partnership program, we only included the Elmira trial, and excluded the Denver and Memphis trial, because in these trials child maltreatment was not included as an outcome measure . The same was true for a second PCIT trial  in which only risk factors for maltreatment were reported, and for an AF-CBT trial  with professional satisfaction and knowledge as outcome. Another reason for exclusion was the use of non-fully random assignment to intervention and control conditions, such as including new participants to the intervention or control condition after randomization was completed. In addition, three studies evaluating the SafeCare  or SEEK program [41, 42] were excluded because clustered randomization was used.
A standardized coding system was used to rate each study on sample, intervention, and design characteristics (Table 2). Background moderators were year of publication and type of publication (journal article or dissertation). Sample characteristics were continent of origin, type of sample (general population, high-risk for maltreatment, maltreating), and ethnicity (majority or minority).
Intervention characteristics consisted of name of the program, focus of the intervention, control condition, location of delivery, delivery format, duration, number of sessions, and age child at onset of intervention. If the content of the intervention and control group were identical , the difference in duration and number of sessions between the intervention and control condition were coded.
Design characteristics included sample size, response rate, use of intent-to-treat analyses (yes/no), blind assessment (yes/no), inclusion of a pre-test measure (yes/no), percentage of participants actively involved in the program by the end of the intervention, and services for the control group. Further, we coded the timing of the post-intervention assessment as the time interval between the end of intervention and the follow-up (in months). We included separate effect sizes for one study if more than one follow-up was reported. For instance, Lam and colleagues  reported child maltreatment outcomes directly after the intervention (time point 0), half a year later (time point 6), and a year after termination of the intervention (time point 12). However, to ensure the independence of samples, only the first follow-up was included to test the overall effect. If results during the intervention as well as results at the end of intervention were reported, only post-test results were included in the meta-analysis. If only results during the intervention were reported, timing of post-intervention assessment was coded as zero and duration of the intervention was adjusted to the timing of assessment. Finally, we coded type of measurement (self-report or other report). If results for more than one type of maltreatment were reported, effect sizes were meta-analytically combined into one effect size, Cohen's d.
All studies were coded independently by two coders. Coders achieved good reliability, intraclass correlations ranged from .65 to 1.00 (M = .96, k = 14 studies), and kappas ranged from .55 to 1.00 (M = .81, 91 % agreement, k = 14 studies). Disagreements were discussed and final scores reflected the consensus of the two coders. To group the interventions based on their focus, four experts independently sorted the 23 intervention programs into four different sets. To obtain a final sorting of the intervention programs, programs clustered by the majority of the experts were considered a category. This final sorting led to four different categories: Support, Parent training, Parent training combined with support, and Other.
The meta-analysis was performed using the Comprehensive Meta-Analysis (CMA) program . For each study, the outcome was transformed into Cohen’s d. Study effect sizes indicate post intervention differences on child maltreatment between the intervention and control group. No outliers were found for study effect sizes. Combined effect sizes were computed using CMA. One outlying sample size  was winsorized, by replacing it with a marginally lower score, while remaining the largest sample size in the set of studies.
Significance tests and moderator analyses were performed through random effects models. In contrast to fixed effect models, random effects models allow for the possibility that there are random differences between studies that are not associated with sampling error and thus point to different study populations [47, 48]. Q-statistics were computed to test the heterogeneity across studies. In addition, we computed 95 % confidence intervals (CIs) around the point estimate of each set of effect sizes. Q-statistics and p-values were also computed to assess differences between combined effect sizes for specific subsets of study effect sizes grouped by moderators. Contrasts were only tested when at least two of the subsets consisted of at least four studies . For continuous moderators, Fisher’s Z-scores were used in weighted least squares meta-regression analyses.
We used the “trim and fill” method [49, 50] to calculate the effect of potential data censoring or publication bias on the outcome of the meta-analysis. Using this method, a funnel plot is constructed of each study’s effect size against the sample size or the standard error (usually plotted as 1/SE, or precision). It is expected that this plot has the shape of a funnel, because studies with smaller sample sizes and larger standard errors have increasingly large variation in estimates of their effect size as random variation becomes increasingly influential, whereas studies with larger sample sizes have smaller variation in effect sizes [50, 51]. The plots are expected to be shaped like a funnel if no data censoring is present. However, since smaller non-significant studies are less likely to be published (the ‘file-drawer’ problem, ), studies in the bottom left hand corner of the plot are often omitted . With the “trim and fill” procedure the k right-most studies considered to be symmetrically unmatched are trimmed and their missing counterparts are imputed or “filled” as mirror images of the trimmed outcomes. This then allows for the computation of an adjusted overall effect size and confidence interval .
Which interventions are most effective?
Twenty different intervention programs were tested in the studies included in the meta-analysis. The effect sizes are shown in a forest plot in Fig. 2. Of the 20 different intervention programs focusing on maltreatment outcomes only five (25 %) programs effectively prevented or reduced child maltreatment (Engaged Moms Program [EMP], ; SOS, ; Multisystemic Therapy for Child Abuse and Neglect [MST-CAN], ; and PCIT with and without individualized services, ). PCIT in its original form  yielded the largest effect size (d = 1.09).
Combined intervention effect
The combined effect size of the 27 intervention effects on maltreatment in the general population, families at-risk for maltreatment or maltreating families was d = 0.13 (N = 4883; 95 % CI: 0.05 ~ 0.21; p < .01), in a heterogeneous set of outcomes (Q = 56.06, p < .01). The trim-and-fill approach showed that 9 studies should be trimmed and filled (Fig. 3), with a resulting non-significant adjusted combined effect size of d = 0.02 (95 % CI: −0.06, 0.11). This pattern of results suggests publication bias favoring the publication of smaller studies with significant findings.
Although no significant combined effect was found, moderator analyses indicated significant differences in effects among subsets of studies. Results of the moderator analyses are shown in Table 3. The moderator analysis for focus of the intervention program showed a significant contrast: programs with a focus on parent training, either with (d = 0.37) or without support (d = 0.37) were significantly more effective than programs that solely provide support (d = 0.03), Q(3) = 15.85, p < .01. Furthermore, interventions with a moderate number of sessions (16–30; d = 0.37) were significantly more effective compared to interventions with fewer (d = 0.05) or more sessions (d = 0.03), Q(2) = 9.65, p < .01. The moderator analysis for duration of the intervention showed comparable results: only interventions with a duration of 6–12 months yielded significant effect sizes (d = 0.23), whereas interventions with a duration shorter than six months (d = 0.22) or longer than twelve months (d = 0.04) did not significantly reduce child maltreatment, Q(2) = 6.04, p < .05. The large majority of the studies (n = 23; 85 %) were conducted in the USA. Because the four studies from outside the USA were all from different countries, the moderating effect of the country of origin of the sample could not be tested. However, no significant contrast was found between samples originating from the USA and samples from other countries. Type of sample was a significant moderator of the effect size. Interventions were significantly more effective in maltreating samples (d = 0.35) than in at-risk samples (d = 0.05), Q(1) = 9.31, p < .01. Moderator analyses for any of the other intervention, sample, or design characteristics did not show significant contrasts.
Meta-regression analyses with one predictor at a time revealed that intervention effects were significantly moderated by year of publication (z = 2.11, p < .05, k = 27) and sample size (z = −2.83, p < .01, k = 27). Studies that were published more recently and studies with smaller sample sizes yielded larger effect sizes. Furthermore, age of the child at the start of the intervention yielded a significant positive regression weight (z = 4.27, p < .001, k = 27), indicating that interventions targeting families with older children had larger effects. Neither socioeconomic status nor ethnicity of the sample significantly influenced the effectiveness of interventions.
We conducted multiple meta-regression analyses to test whether the moderating effects of focus, type of sample, age of the child, number of sessions or duration of the intervention were confounded by the background characteristics year of publication and/or sample size. In the first model, we included year of publication and sample size. Both year of publication (z = 2.53, p < 0.05, k = 27) and sample size (z = −2.83, p < .01, k = 27) were significant predictors of the effect size. Next, the five significant moderators were added separately to the model. Focus was added as a dichotomous variable, indicating the difference between support only and training with or without support. When controlled for the two background variables, program focus was a significant predictor of intervention effectiveness (z = 3.00, p < .01, k = 23). Interventions that provide parent training yielded larger effect sizes compared to interventions that provide only support. Type of sample was not significant (z = 1.90, p = .06, k = 26), indicating that controlling for year of publication and sample size, interventions targeting maltreating samples did not yield larger effect sizes compared to at risk samples.
The effect of child age at the start of the intervention failed to be a significant moderator when year of publication and sample size were taken into account (z = 1.74, p = .08, k = 27). In the next regression, number of sessions and duration of the intervention were both added as two dummy variables. The first dummy indicated the difference between interventions in the first category (< 16 sessions or < 6 months) versus the other two categories, and the second dummy indicated the difference between the middle category (16–30 sessions or 6–12 months) versus the first and last category. On top of the two background characteristics, neither the duration (ps > .14) nor the number of sessions of the intervention (ps > .14) significantly predicted the effect size.
Long term effects
Six studies included more than one follow-up. To examine the possible long term effects of interventions, the difference between the effect sizes at the first and last follow-up was calculated, such that a positive difference indicated a larger effect at the last assessment compared to the first assessment. The combined effect size for the six difference scores was not significant (d = 0.13 [N = 530; 95 % CI: −0.04 ~ 0.31; p = 0.14], in a homogeneous set of outcomes [Q = 2.34, p = .80]), indicating that the effect of interventions on reducing or preventing child maltreatment did not change over time. However, this finding is based on only six studies and should therefore be considered as exploratory.
Contrary to our expectations, the current meta-analysis did not show significant combined effects of intervention programs in randomized controlled trials on the reduction or prevention of child maltreatment in the general population, at-risk or maltreating families. Taking into account the presence of publication bias against smaller studies with non-significant results in this research domain, we failed to find a significant overall effect.
We did find, however, a subset of studies with promising intervention effects. First of all, differences between intervention studies in effect sizes could be partly explained by the focus of the intervention program. Programs that solely provide support, such as promoting healthy behaviors during pregnancy and early parenthood , establishing social support networks , or screening for developmental delay  were not effective. In contrast, we found significant effects for intervention programs offering parent training, regardless whether additional support is provided or not. This subgroup of intervention programs includes, among others, Multisystemic Therapy for Child Abuse and Neglect  and Parent Child Interaction Therapy (with and without additional individualized services; ), which were both effective in reducing child maltreatment. Secondly, we found significant intervention effects in maltreating samples, but not in at-risk samples, indicating that programs are only effective in reducing (but not preventing) child maltreatment. Although previous reviews have suggested limited effectiveness of programs in reducing child maltreatment (e.g., [58, 59]), the current study is, to our knowledge, the first to meta-analytically compare programs to prevent and reduce child maltreatment.
The 'less is more' effect in attachment-based interventions found by Bakermans-Kranenburg and colleagues  seems only partly applicable to programs aimed at reducing or preventing child maltreatment. We found a curvilinear association with program duration and number of program sessions. Programs with a moderate duration (6–12 months) or a moderate number of sessions (16–30) yielded significantly higher effect sizes compared to shorter or longer programs and programs with fewer or more sessions. This indicates that for at-risk or maltreating parents who are targeted in these interventions somewhat more comprehensive programs with longer duration may be needed in order to effectively change parenting behavior. At the same time, our results indicate that programs should not provide services for too long. For example, a recent RCT ([35, 38]; not included in the meta-analysis, because maltreatment was not an outcome measure) indicated that standard PCIT with 12 sessions was equally effective or even significantly more effective for at-risk and maltreating families compared to time-variable PCIT with on average 17 sessions. More is not necessarily better.
Although the effectiveness of intervention programs is sometimes presented as promising (e.g., [1, 3]), the current meta-analytic results prove the opposite and seem to support a rather gloomy picture of the evidence base of widely used intervention programs. Several factors may contribute to this contrast. First, the main outcome measures in many effectiveness studies aimed at preventing child maltreatment are risk factors for child maltreatment (e.g., Child Abuse Potential Inventory, parenting stress), instead of actual maltreatment measures. Although risk factors are important and the reduction of these factors following the intervention program should be considered a positive sign, they do not provide information about the actual prevention or reduction of child maltreatment which is the goal of the interventions. For example, in the current meta-analysis, nearly 20 % of full text papers were excluded because child maltreatment was not an outcome measure. Moreover, some interventions were effective in improving parenting or child health, but did not effectively prevent or reduce child maltreatment. For instance, Jouriles and colleagues  found positive effects of Project Support on several problematic parenting variables, such as mothers' perceived inability to manage childrearing responsibilities, psychological distress, and observed ineffective parenting, whereas the number of subsequent reports to CPS did not differ between the experimental and control condition. Second, in some RCTs, effects were found in specific subgroups, which were not fully randomized as such. For example, Olds and colleagues  conclude that their Nurse-Family Partnership has a significant effect on child maltreatment at direct follow-up for pregnant women with each of three risk factors. However, the original randomized sample included pregnant women of whom 85 % had at least one of the three risk factors and only 23 % had each of the three risk factors. There was no significant effect when the whole sample was included in the analyses. Such subgroup analyses indicate that the results are no longer based on a true RCT, and therefore may yield inflated effect sizes that have to be replicated in independent trials randomized with stratification across number of risk factors.
Sample size and year of publication were both significant and independent predictors of the effect size in a multivariate approach. Effect sizes were smaller with increasing sample size and larger effect sizes were found for later studies, which may indicate that intervention programs improve or become more fine-tuned over time. Meta-analytic findings from Piquart and Teubert  indicated the opposite effect, with later studies yielding smaller effect sizes. This result was different from what the authors expected, and they argued that the general knowledge about parenting has increased over time, which reduces the size of possible intervention effects. In addition, they suggest that publication bias towards significant findings was a problem in the earlier years. The fact that we did find a positive effect for year of publication may be caused by the fact that we included studies that were published up until 2013, whereas Piquart and Teubert  only included studies published up until 2009. After controlling for sample size and year of publication only focus of the program remained a significant predictor of program effectiveness.
The current meta-analysis is based on a rather small number of studies. Part of the moderator analyses should therefore be considered as exploratory. Whereas the full sample consisted of a set of 27 effect sizes, several moderator analyses were based on a relatively small number of effect sizes. For example, only six studies included more than one follow-up. More long term follow-ups are needed to obtain more robust findings about possible effects of programs to prevent and reduce child maltreatment on the long term.
In addition, some characteristics lacked variance to test for moderating effects. Given the small number of RCTs conducted outside the USA, the possibility to examine the moderating effect of continent of origin was limited. In the same vein, it was not possible to examine the effect of type of intervener, since all intervention programs that were included in the current meta-analysis were delivered by (para)professionals and none by other interveners such as laymen. Furthermore, programs targeting the general population were excluded from moderator analyses, since only one such study was included in the meta-analysis. Finally, many studies included in the meta-analysis only included an overall measure of maltreatment or only one type of maltreatment, mostly physical abuse. Therefore, it was impossible to examine program effectiveness on different types of maltreatment. In order to disentangle the effect of intervention programs more thoroughly, and thus to provide the most appropriate services to different families, RCTs should separately report on different types of maltreatment.
This meta-analysis implies several possibilities for improvement of program effectiveness studies. First, we clearly need more RCTs that examine the effect of intervention programs on the prevention or reduction of child maltreatment, also outside the USA and in low- and middle-income countries. Nearly half of all full text papers that were screened for eligibility had to be excluded because no RCT was done. Indeed, it has been argued that it is too difficult for practical and ethical reasons to conduct RCTs with maltreating or at-risk families , and this may be even more so for low- and middle-income countries. However, the fact that we did find six RCTs in maltreating families and 20 RCTs in at-risk families, including one study in Iran, indicates that it is possible to carry out rigorous effectiveness studies in various populations. Only RCTs can strengthen the evidence base for maltreatment interventions needed so badly in practice.
Moreover, effectiveness studies of intervention programs that aim to reduce or prevent child maltreatment should take child maltreatment as their primary outcome measure. As noted above, many studies mainly focus on parenting behaviors or risk factors for child maltreatment, while actual changes in child maltreatment seem somewhat neglected. When child maltreatment measures are used as main outcome variables, a multimethod approach is advisable (e.g., ). Previous meta-analytic evidence indicates that reports from professionals yield considerably lower prevalence rates than self-report measures . Therefore, effect sizes of intervention programs will be more reliable when more than one method is used to examine how often child maltreatment occurs . In addition, for intervention programs that do effectively reduce child maltreatment, future studies should examine whether they are also effective in reducing the number and duration of out-of-home placements.
Finally, future research should focus on the development and testing of prevention programs. Results of our meta-analysis indicate that so far intervention programs are only effective in reducing child maltreatment, and thus only protect children when the harm has been done. From all programs in the current meta-analysis that aim to prevent child maltreatment, only one program was universal , while 19 other programs targeted a high-risk population. However, prevention efforts should not only focus on populations with the highest risk for maltreatment. From all maltreating families, only a small proportion belongs to this group of high risk families . In order to protect all children against maltreatment, the development and testing of both universal and targeted prevention programs should be prioritized.
Findings from the current meta-analysis indicate that intervention programs have no overall significant effects on the reduction or prevention of child maltreatment, when publication bias against smaller studies with non-significant results is taken into account. Significant effects were however found for intervention programs reducing child maltreatment in maltreating families and for interventions that provide parent training. More RCTs are needed to strengthen the evidence base of program effectiveness, especially in the field of prevention.
*Publications marked with an asterisk were included in the meta-analysis.
Daro D. Prevention of child abuse and neglect. In: Meyers JEB, editor. The APSAC handbook on child maltreatment. 3rd ed. Thousand Oaks, CA: SAGE Publications, Inc; 2011. p. 17–37.
Carr A. The evidence base for family therapy and systemic interventions for child-focused problems. J Fam Ther. 2004;36:107–57.
Mikton C, Butchart A. Child maltreatment prevention: A systematic review of reviews. Bull World Health Organ. 2009;87:353–61.
Stoltenborgh M, Bakermans-Kranenburg MJ, Alink LRA, Van IJzendoorn MH. The prevalence of child maltreatment across the globe: Review of a series of meta-analyses. Child Abuse Rev. 2014;24:37–50.
Cicchetti D, Valentino K. An ecological transactional perspective on child maltreatment: Failure of the average expectable environment and its influence upon child development. In: Cicchetti D, Cohen DJ, editors. Developmental psychopathology: Risk, disorder, and adaptation (2nd ed., Vol. 3). New York, NY: John Wiley; 2006. p. 129–201.
Euser S, Alink LRA, Pannebakker F, Vogels T, Van IJzendoorn MH, Bakermans-Kranenburg MJ. The Prevalence of Child Maltreatment in the Netherlands across a 5-Year Period. Child Abuse Negl. 2013;37:841–51.
Stith SM, Liu T, Davies LC, Boykin EL, Alder MC, Harris JM, et al. Risk factors in child maltreatment: A meta-analytic review of the literature. Aggress violent behav. 2009;14:13–29.
Alink LRA, Cicchetti D, Kim J, Rogosch FA. Longitudinal associations among child maltreatment, social functioning, and cortisol regulation. Dev Psychol. 2012;48:224–36.
Gilbert R, Spatz Widom C, Browne K, Fergusson D, Webb E, Janson S: Child Maltreatment 1: Burden and consequences of child maltreatment in high-income countries. Lancet 2008; doi:10.1016/S0140-6736(08)61706-7.
Jonson-Reid M, Kohl PL, Drake B. Child and adult outcomes of chronic child maltreatment. Pediatrics. 2012;129:839–45.
Trickett PK, Noll JG, Putnam FW. The impact of sexual abuse on female development: Lessons from a multigenerational, longitudinal research study. Dev Psychopathol. 2011;23:453–76.
Nanni V, Uher R, Danese A. Childhood maltreatment predicts unfavorable course of illness and treatment outcome in depression: A meta-analysis. Am J Psychiatr. 2012;169:141–51.
Sanders MR. Triple P Parenting Program: Toward an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clin Child Fam Psychol Rev. 1999;2:71–90.
Oveisi S, Ardabili HE, Dadds MR, Majdzadeh R, Mohammadkhani P, Rad JA, et al. Primary prevention of parent–child conflict and abuse in Iranian mothers: A randomized-controlled trial. Child Abuse Negl. 2010;34:206–13.
Olds DL, Henderson CR, Chamberlin R, Tatelbaum R. Preventing child abuse and neglect: A randomized trial of nurse home visitation. Pediatrics. 1986;78:65–78.
Olds DL. The Nurse-Family Partnership: An evidence-based preventive intervention. Infant Mental Health Journal. 2006;27:5–25.
Olds DL, Henderson CR, Kizman H. Does prenatal and infancy nurse home visitation have enduring effects on qualities of parental caregiving and child health at 25 to 50 months of life? Pediatrics. 1994;93:89–98.
Eyberg SM, Robinson EA. Parent–child Interaction Training: Effect on family functioning. J Clin Child Psychol. 1982;11:130–7.
Runyon MK, Urquiza AJ. Child physical abuse: Interventions for parents who engage in coercive parenting practices and their children. In: Meyers JEB, editor. The APSAC handbook on child maltreatment. 3rd ed. Thousand Oaks, CA: SAGE Publications, Inc; 2011. p. 17–37.
Chaffin M, Silovsky JF, Funderburk B, Valle LA, Brestan EV, Balachova T, et al. Parent–child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. J Consult Clin Psychol. 2004;72:500–10.
Layzer JI, Goodson BD, Bernstein L, Price C. National evaluation of family support programs. Vol. A: The meta-analysis. Final report. Cambridge, MA: Abt Associates; 2001.
Pinquart M, Teubert D. Effects of parenting education with expectant and new parents: A meta-analysis. J Fam Psychol. 2010;24:316–27.
Filene JH, Kaminski JW, Valle LA, Cachat P. Components associated with home visiting program outcomes: A meta-analysis. Pediatrics. 2013;132:S100–9.
Geeraert L, Van den Noortgate W, Grietens H, Onghena P. The effects of early prevention programs for families with young children at risk for physical child abuse and neglect: A meta-analysis. Child Maltreat. 2004;9:277–91.
Guterman NB. Enrollment strategies in early home visitation to prevent physical child abuse and neglect and the “universal versus targeted” debate: A meta-analysis of population-based and screening-based programs. Child Abuse Negl. 1991;23:863–90.
Sweet MA, Appelbaum MI. Is home visiting an effective strategy? A meta-analytic review of home visiting programs for families with young children. Child Dev. 2004;75:1435–56.
Healthy Families America. Healthy Families America critical elements. Chicago, IL: Prevent Child Abuse America; 2001.
Jouriles EN, McDonald R, Rosenfield D, Norwood WD, Spiller L, Stephens N, et al. Improving parenting in families referred for child maltreatment: A randomized controlled trial examining effects of Project Support. J Fam Psychol. 2010;24:328–38.
Brayden RM, Altemeier WA, Dietrich MS, Tucker DD, Christensen MJ, McLaughlin FJ, et al. A prospective study of secondary prevention of child maltreatment. J Pediatr. 1993;122:511–6.
Swenson CC, Schaeffer CM, Henggeler SW, Faldowski R, Mayhew AM. Multisystemic therapy for child abuse and neglect: A randomized effectiveness trial. J Fam Psychol. 2010;24:497–507.
Bakermans-Kranenburg MJ, Van IJzendoorn MH, Juffer F. Less is more: Meta-analyses of sensitivity and attachment interventions in early childhood. Psychol Bull. 2013;129:195–215.
Olds DL, Sadler L, Kitzman H. Programs for parents of infants and toddlers: recent evidence from randomized trials. J Child Psychol Psychiatry. 2007;48:335–91.
Bakermans-Kranenburg MJ, Van IJzendoorn MH, Pijlman FTA, Mesman J, Juffer F. Experimental evidence for differential susceptibility: Dopamine receptor polymorphism (DRD4 VNTR) moderates intervention effects on toddlers’ externalizing behavior in a randomized control trial. Dev Psychol. 2008;44:293–300.
Creighton, SJ: Recognising changes in incidence and prevalence. In: Early prediction and prevention of child abuse. A handbook. Edited by Browne KD, Hanks H, Stratton P, Hamilton C. Chichester: Wiley; 2002.
Thomas R, Zimmer-Gembeck MJ. Accumulating evidence for Parent–child interaction therapy in the prevention of child maltreatment. Child Dev. 2011;82:177–92.
Prinz RJ, Sanders MR, Shapiro CJ, Whitaker DJ, Lutzker JR. Population-based prevention of child maltreatment: The U.S. Triple P system population trial. Prev Sci. 2009;10:1–12.
Olds DL. Preventing child maltreatment and crime with prenatal and infancy support of parents: The nurse-family partnership. Journal of Scandinavian Studies in Criminology and Crime Prevention. 2008;9:2–24.
Thomas R, Zimmer-Gembeck MJ. Parent–child interaction therapy: An evidence-based treatment for child maltreatment. Child Maltreat. 2012;17:253–66.
Kolko DJ, Baumann BL, Herschell AD, Hart JA, Holden EA, Wisniewski SR. Implementation of AF-CBT by community practitioners serving child welfare and mental health: A Randomized trial. Child Maltreat. 2012;17:32–46.
Chaffin M, Hecht D, Bard D, Silovsky JF, Beasley WH. A statewide trial of the SafeCare home-based services model with parents in child protective services. Pediatrics. 2012;129:509–15.
Dubowitz H, Feigelman S, Lane W, Jeongeun K. Pediatric primary care to help prevent child maltreatment: The Safe Environment for Every Kid (SEEK) model. Pediatrics. 2009;123:858–65.
Dubowitz H, Lane WG, Semiatin JN, Magder LS. The SEEK model of pediatric primary care: Can child maltreatment be prevented in a low-risk population? Acad Pediatr. 2012;12:259–68.
DePanfilis D, Dubowitz H. Family Connections: A program for preventing child neglect. Child Maltreat. 2005;10:108–23.
Lam WKK, Fals-Stewart W, Kelley ML. Parent training with behavioral couples therapy for fathers’ alcohol abuse: Effects on substance use, parental relationships, parenting, and CPS involvement. Child Maltreat. 2009;14:243–54.
Borenstein M, Rothstein D, Cohen J. Comprehensive meta-analysis: A computer program for research synthesis [Computer software]. Biostat: Englewood, NJ; 2005.
DuMont K, Mitchell-Herzfeld S, Greene R, Lee E, Lowenfels A, Rodriguez M, et al. Healthy Families New York (HFNY) randomized trial: Effects on early child abuse and neglect. Child Abuse Negl. 2008;32:295–315.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods. 2010;1:97–111.
Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks, CA: Sage; 2001.
Duval S, Tweedie R. A nonparametric ‘trim-and-fill’ method of assessing publication bias in meta-analysis. J Am Stat Assoc. 2000;95:89–8.
Duval S, Tweedie R. Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56:455–63.
Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR. Empirical assessment of effect of publication bias on meta-analyses. Br Med J. 2000;320:1574–7.
Mullen B. Advanced BASIC meta-analysis. Hillsdale, NJ: Lawrence Erlbaum; 1989.
Gilbody SM, Song F, Eastwood AJ, Sutton AJ. The causes, consequences and detection of publication bias in psychiatry. Acta Psychiatr Scand. 2000;102:241–9.
Dakof GA, Cohen JB, Henderson CE, Duarte E, Boustani M, Blackburn A, et al. A randomized pilot study of the Engaging Moms Program for family drug court. J Subst Abuse Treat. 2010;38:263–74.
Olds DL, Eckenrode J, Henderson CR, Kitzman H, Powers J, Cole R, et al. Long-term effects of home visitation on maternal life course and child abuse and neglect: Fifteen-year follow-up of a randomized trial. J Am Med Assoc. 1997;278:637–43.
Bugental DB, Ellerson PC, Lin EK, Rainey B, Kokotovic A. A cognitive approach to child abuse prevention. J Fam Psychol. 2002;16:243–58.
Duggan A, Caldera D, Rodriguez K, Burrell L, Rohde C, Crowne SS. Impact of a statewide home visiting program to prevent child abuse. Child Abuse Negl. 2007;31:801–27.
Fraser JG, Lloyd SW, Murphy RA, Crowson MM, Casanueva C, Zolotor A, et al. Comparative effectiveness of interventions addressing maltreatment. In: Rockville (MD): Agency for Healthcare Research and Quality (US); Comparative Effectiveness Reviews, No 89. 2013. http://www.ncbi.nlm.nih.gov/books/NBK137808/. Accessed 7 Sept 2015.
MacMillan HL, Wathen CN, Barlow J, Fergusson DM, Leventhal JM, Taussig HN. Interventions to prevent child maltreatment and associated impairment. Lancet. 2009;373:250–66.
Van IJzendoorn MH, Bakermans-Kranenburg MJ, Juffer F. Why less is more: From the Dodo bird verdict to evidence-based interventions on sensitivity and early attachments. In: Berlin LJ, Ziv Y, Amaya-Jackson L, Greenberg MT, editors. Enhancing early attachments: Theory, research, intervention, and policy. NY: The Guilford Press; 2005. p. 297–312.
Shaffer A, Huston L, Egeland B. Identification of child maltreatment using prospective and self-report methodologies: A comparison of maltreatment incidence and relation to later psychopathology. Child Abuse Negl. 2008;32:682–92.
Alink LRA. Kindermishandeling beter in beeld. Inaugural lecture. Amsterdam: Vrije Universiteit Amsterdam; 2013.
Barth RP. An experimental evaluation of in-home child abuse prevention services. Child Abuse Negl. 1991;15:363–75.
Bugental DB, Schwartz A. A cognitive approach to child mistreatment prevention among medically at-risk infants. Dev Psychol. 2009;45:284–8.
Bybee D: The family diversion project: Experimental evaluation of an intervention for abusing and neglecting families (Unpublished doctoral dissertation). Michigan: Michigan State University, Department of Psychology; 1985.
Chambliss JW: An experimental trial of a home visiting program to prevent child maltreatment (Unpublished doctoral dissertation). Atlanta, Georgia: Georgia State University; 1998.
Duggan A, McFarlane E, Fuddy L, Burrell L, Higman SM, Windham A, et al. Randomized trial of a statewide home visiting program: Impact in preventing child abuse and neglect. Child Abuse Negl. 2004;28:597–622.
Fergusson DM, Grant H, Horwood LJ, Ridder EM. Randomized trial of the Early Start Program of home visitation. Pediatrics. 2005;116:e803–9.
LeCroy CW, Krysik J. Randomized trial of the healthy families Arizona home visiting program. Children Youth Serv Rev. 2011;33:1761–6.
MacMillan HL, Thomas BH, Jamieson E, Walsh CA, Boyle MH, Shannon HS, et al. Effectiveness of home visitation by public-health nurses in prevention of the recurrence of child physical abuse and neglect: A randomized controlled trial. Lancet. 2005;365:1786–93.
McIntosh E, Barlow J, Davis H, Stewart-Brown S. Economic evaluation of an intensive home visiting programme for vulnerable families: A cost-effectiveness analysis of a public health intervention. J Public Health. 2009;31:423–33.
Silovsky JF, Bard D, Chaffin M, Hecht D, Burris L, Owora A, et al. Prevention of child maltreatment in high-risk rural families: A randomized clinical trial with child welfare outcomes. Children Youth Serv Rev. 2011;33:1435–44.
Stevens-Simon C, Nelligan D, Kelly L. Adolescents at risk for mistreating their children Part II: A home- and clinic-based prevention program. Child Abuse Negl. 2001;6:753–69.
The study was supported by the Netherlands Organization for Scientific Research (MJBK: VICI Grant no. 453-09-003; MHvIJ: NWO SPINOZA prize).
The authors declare that they have no competing interests.
SE carried out the literature search and coding of studies, and drafted the manuscript. SE and MHvIJ performed the analyses. MS carried out the literature search and coding of studies. LRA, MJBK and MHvIJ conceived of the study, contributed to the study design, analysis, interpretation of data, and drafting or revision of the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Euser, S., Alink, L.R., Stoltenborgh, M. et al. A gloomy picture: a meta-analysis of randomized controlled trials reveals disappointing effectiveness of programs aiming at preventing child maltreatment. BMC Public Health 15, 1068 (2015). https://doi.org/10.1186/s12889-015-2387-9
- Child maltreatment