The results suggest that treatment was not particularly effective. The following lines of evidence point to this conclusion. Correlations between treatment attendance and outcome were very small (as shown in Table 2). A median 3% of the variance in outcome might be attributed to treatment.
The correlations existed before most treatment occurred, at week 1 (Table 3). We would normally infer from the correlations in Table 2 that more treatment produces better drinking outcomes, but the Table 3 correlations suggests the reverse, that better drinking levels predict more treatment.
Nearly two thirds of the long term improvement in the full treatment group was matched by the untreated rapid dropout group (Figures 1 &2 and Table 6). Only in the remaining one third could there be a subcomponent consisting of a treatment effect.
Most of the improvement was instantaneous, occurring at week 1, before the participants had received the bulk of their treatment (Figures 3 &4 and Table 6). Although the full treatment group received 11 more therapy sessions, the additional improvement was of small magnitude. For example, at week one percent days abstinence had increased by over 60%, and the additional 11 weeks of treatment increased it by only 4%. If treatment were the causal agent we would expect that the effect would occur over the course of weeks with the administration of treatment.
There was a similar instantaneous improvement in untreated alcoholics (Figures 3 &4 and Table 6). The effect size estimates suggested that nearly three fourths of the instantaneous improvement in the full treatment group was matched by the untreated group.
Those who received zero treatment sessions had better outcomes than those who received one session (Figures 1 to 4 and Tables 4 to 6). The implication of this is discussed below.
Improvement was maintained over time even in the no treatment group (Figures 1 to 4 and Table 6). Change from week 1 to follow-up was not significantly different between the zero and full treatment groups. In both groups the week 1 to week 12 improvement was lost by follow-up. These data do not support the contention that retention of clients in treatment for as long as possible increases the chances that they will derive benefit from therapy.
A more reasonable interpretation of these data is that they illustrate the importance of selection effects, i.e., participants who reduce their alcohol consumption are more likely to enter or remain in treatment and those who continue drinking are more likely to drop out of treatment. One of the best studies of alcoholism treatment outcome was conducted by the Rand Corporation in the late 1970s [10]. Participants were patients who attended inpatient or outpatient treatment at centers across the United States. They found that "it is possible that the correlation [between attendance and outcome] arises from selection effects, such that the better motivated or more successful patients continue in treatment, whereas the more intractable cases drop out. Such a pattern could result from subject self-selection or from the operation of the treatment environment in encouraging continued participation for more responsive patients." (page 155). The likely selection effect in the current data was illustrated by the anomalous group participants who dropped out after attending only one treatment session as shown in Figures 1 and 2. Those who received zero treatment had better outcomes than those who received one session of treatment. Few would argue that this shows that treatment was harmful. A more likely explanation for this difference is some sort of self-selection. The higher drinking level of the 1 session dropout participants at baseline suggests that they may have been more dependent on alcohol than those in the 0 session dropout group. The relative higher level of dependence may have put these individuals under more pressure to do something about their drinking, explaining why they did not drop out prior to the first session. A similar logic could apply to the outcomes of the consistent attendees of the full treatment group. The likely selection effect is also shown in that participants with higher drinking levels at week 1 were more likely to drop out of treatment (Table 3).
The decreased drinking in both untreated and treated participants can be explained by a number of factors. One factor is that part of the effect is not real; many active alcoholics underreport drinking. Collateral informant interviews and other verification techniques are only partially effective in correcting the data. The Rand study [10], for example, found that 30% of the collateral informants were unable to provide information. Underreporting can make treatment appear more effective than it actually is.
Additionally, there are a number of non-treatment effects likely to result in reduced drinking [19, 21–23]. In order to enter the trial participants had to first achieve a level of abstinence or reduced intake. If a participant arrives at a site in an intoxicated state immediate action is required by staff, such as admission to a detox unit, or detainment in the waiting room until the breath alcohol level returns to normal. These rules would have applied to each participant in Project MATCH at the time of enrollment and would have contributed to the rapid improvement seen in the week one data. The pre-study screening procedures used in clinical trials, both the overt criteria and the subjective criteria, are designed to select participants who are motivated to reduce their drinking. Enrolling in the trial suggests that the alcoholic has crystallized a decision to reduce or abstain from drinking. Once in the trial, the continued monitoring of drinking behavior by staff personnel may have both motivational and therapeutic benefits. For example, in one study with a 2 year follow-up [21], over half the participants indicated they liked the "caring, concern and help" follow-up telephone contact, and in another [24], the telephone interviewers reported that they usually entered in a sympathetic interaction with the study participants. Such positive empathetic contact could be of therapeutic benefit.
There are a number of limitations to these analyses. The data from two thirds of the subjects were not used in the illustration of mean drinking levels shown in the figures and in Table 6. However these data were used in Tables 2 though 5, and are presented in more detail in the attached data [see Additional File 1]. The linear relationship shown in Tables 2 and 3, and the means in Tables 4 and 5 indicated that no information was left out. Additionally, other analyses of these groups have been previously published, e.g., by the Project MATCH research group [20]. We chose to highlight several specific groups. The logic of selecting the group that received no treatment and the group that received all 12 sessions of treatment was clear – they offered an unambiguous treatment comparison. The data presented here show that the outcomes of the 12 session group were better than the outcomes of participants who received between 2 and 11 sessions, making the 12 session group a fair comparison. Additionally we identified an anomalous group, the one session rapid dropouts, and used that group in an attempt to interpret the data. The mathematical bases for the anomalous designation, and thus the selection of this group, were presented.
Analyses were primarily limited to descriptive and simple inferential statistics. This was done because the findings are likely to be extremely controversial. We have therefore presented results that are easily replicated, and easily understood.
Although the results are essentially negative, suggesting that current treatments are not effective, we do not offer suggestions for future directions. We feel we will have made a contribution if the data presented can be accepted as accurate. If they are accepted then implications for future research and treatment will naturally follow. For example, if the patient's motivations, opportunities, beliefs and hopes are the critical issues, how do we measure them? How do we influence them? How do they interact with the treatment environment?
It may be that pre-treatment patient characteristics (e.g., level of dependence, social support, etc.) have a large influence on both the number of treatment sessions attended and drinking outcome. However, even if this is true, it would not be evidence of treatment effectiveness. Only if one could show that positive prognostic factors were weighted heavily against the treatment attendees and in favor of the dropouts would these results be open to reinterpretation. The baseline and week one drinking data presented here do not support the likelihood of such a possibility. Additionally, there is no evidence in the literature to support the notion that, for example, alcoholics who lack social support are more likely to enter or remain in treatment. There are a large number of both positive and negative reasons why alcoholic participants drop out of clinical trials. Positive reasons include work commitments, pregnancy, re-location to another area and remission from drinking. Negative reasons include continued or increased drinking, abuse of other substances, attitude towards the clinical staff or environment, physical illness, hospitalization and incarceration.
Conclusions drawn from therapy delivered in clinical trials might not be applicable to therapy in other settings. We might well expect great differences in clinical effectiveness between different therapists, and between different treatment programs. However it can be argued that the large non-treatment effect seen in this study is present in other aggregated outcome studies published in the literatures. Miller and others [9] presented results from a number of such trials, in addition to Project MATCH. Table 1 summarized their findings for the two outcome variables studied here. The outcomes of the different studies are remarkably similar. The similarity in results would suggest that the non-treatment effect identified here may be present in all these studies.
The outcome variables in these analyses were the original primary MATCH outcome variables. We have been able to show that the analyses of these variables, and the treatment attendance variable, are in perfect concordance with published analyses of the Project MATCH Research group [20]. Over 60 publications have been generated by Project MATCH, but, to the best of our knowledge, all have overlooked the main finding of this study, i.e., the good outcomes of the zero treatment group when compared to the full treatment group and that the improvement in all groups occurred immediately after enrollment in the trial. Ineffective treatment would be the most parsimonious explanation for the rather surprising main findings of Project MATCH, that there was no match between patient characteristics and different types of treatment, and that all three treatments were equal.
There may be similarities between these results, for alcoholism patients, and effects seen in some other types of patients. Depressed patients sometimes report significant improvement after enrolling in clinical trials but before receiving therapy [25]. Recent time-course analyses in depression report sudden decreases in depression regardless of treatment condition [26]. These rapid responders were associated with better outcome at the end of the treatment and into follow-up [27].
It is difficult to compare the high quality follow-up data of Project MATCH to that in the alcoholism literature, much of which are collected under quite different circumstances. The zero treatment participants at the final follow-up interval (month 15) reported a mean of 25.1 drinks per week, with 45% (35/78) abstinent. These outcomes appear somewhat better than those recently summarized in the literature [17]. Of some 17 studies than included placebo or no treatment conditions, with and without prior detoxification, a mean (for studies) was 21% abstinent, and the average participant was drinking 31 drinks per week [17].
Exaggerated claims of treatment effectiveness can have undesirable consequences for patients, for therapists, and for science. Patients who fail an "effective" treatment may feel even more hopeless. This increased despair may be extremely deleterious in people with such life-threatening habits. Therapists may feel inadequate or frustrated with repeated failures. For science, exaggerated claims tend to shift focus into unproductive directions, and to obscure the pertinent facts that are necessary in order to move the science forward.
While this study shows that three of the best treatments currently available for addiction were not very effective, it remains likely that many severely dependent alcoholic individuals benefit from external help. By suggesting practical and helpful ways for dealing with the problems of addiction, therapy may help a patient regain a sense of control over his or her life. We are not suggesting that alcoholism treatment should be discontinued or even reduced. People with alcohol problems clearly need all the help our society can give them.
Comments
View archived comments (1)