An exploratory review of HIV prevention mass media campaigns targeting men who have sex with men

Background Men who have sex with men (MSM) are at increased risk of HIV infection in both high- and low-income settings. Mass media campaigns have been used as a means of communicating HIV health promotion messages to large audiences of MSM. There is no consensus on which designs are most appropriate to evaluate the process and outcomes of such interventions. Methods An exploratory review was conducted to assess research examining awareness, acceptability, effects on HIV testing, disclosure and sexual risk, and cost-effectiveness of HIV mass media campaigns targeting MSM. We searched for quantitative and qualitative studies published between 1990 and May 2011 via the Cochrane Central Register of Controlled Trials, MEDLINE, EMBASE, Psych Info, ISI Web of Science, OpenGrey and COPAC, and contacting experts. No exclusions were made on the basis of study design or methods because our primary aim was to map evidence. We appraised study quality and present a narrative synthesis of findings. Results Sixteen reports from 12 studies were included. All were from high-income countries and most examined multi-media interventions. Half of the studies were single cross-sectional surveys. Three repeat cross-sectional studies collected data pre and post the campaign launch. The remaining three studies monitored routine data. Three studies included a nested qualitative component. Campaign coverage was the most commonly reported outcome (9 studies). Imagery, tone of language, content and relevance were identified in the qualitative research as factors influencing campaign acceptability. HIV testing rates (or intention to test) were reported by five studies. Two studies reported that testing rates were higher among men who had seen the campaigns compared to men who had not, but this may reflect confounding. Findings were less consistent regarding reductions in sexual risk behaviours (4 studies). None of the studies examined cost-effectiveness. Conclusions Campaigns aim to provide MSM with information to help prevent transmission of HIV and to address increasing motivation and changing norms towards precautionary behaviours. However, the limitations of mass media in imparting skills in effecting behaviour change should be recognised, and campaigns supplemented by additional components may be better-suited to achieving these goals.


Background
Worldwide, sex between men accounts for between 5 and 10% of HIV infections, but the proportion is far higher in much of the developed world where sex between men is the most common means of transmission [1]. Sex between men is also a prominent feature in the epidemiology of HIV in other regions such as Latin America where men who have sex with men (MSM) are at increased risk of HIV infection.
Mass media interventions have the potential to reach large audiences, providing them with information and raising awareness. Campaigns can also present role models and aim to change normative beliefs, as well as helping put health issues on policy-makers' agenda [2]. Mass media interventions can potentially reach individuals or groups who may not be accessing other statutory or community-based interventions. They may use broadcast media, such as television, radio or film; print media, such as posters and newspapers; outdoor media, such as billboards; or digital media, such as the internet [3].
Although HIV prevention mass media campaigns have been criticised for using weak evaluation design [4,5], there is no consensus on which designs are most appropriate for evaluating process and outcomes [6]. While randomised controlled trials (RCTs) provide the most rigorous means of evaluating public health interventions, they are not generally applicable to mass media interventions because of obvious challenges concerning contamination or the lack of sufficient units for statistical power [2]. Alternatives include interrupted time-series analyses comparing repeat data on outcomes from a population before and after exposure, and crosssectional studies comparing outcomes among exposed and unexposed individuals. Which of these provide the least biased estimates of effect is likely to depend on context; for example intervention coverage, secular trends in outcomes and likely effect sizes [6]. The evaluation of the effects of mass media interventions on MSM is further complicated by the lack of a sampling frame for this population necessitating convenience samples [7]. Furthermore, other questions of importance to policy-makers such as intervention coverage and acceptability are also not amenable to experimental designs, and in the case of acceptability may be examined via quantitative and qualitative research.
A Cochrane systematic review conducted by Vidanapathirana et al. in 2005 assessed the effects of mass media on HIV testing among the general population and specific target groups, including MSM [8]. The authors concluded mass media campaigns were effective in increasing testing in the general population in the short-term, although no long-term impacts on HIV testing were observed. However, only one of the studies included in the review targeted MSM [9].
Given the lack of previous reviews of HIV prevention mass media interventions targeting MSM, we aimed to examine literature in this area. Given our interest in examining questions of awareness and acceptability as well as effectiveness and cost-effectiveness, and given the lack of consensus on which designs are most appropriate, our exploratory review aimed to systematically map evidence in this area, appraise its quality, and narratively synthesise its findings. The following research questions (RQ) are examined: (RQ1) How successfully do HIV prevention mass media interventions achieve awareness among their target audience(s) of MSM? (RQ2) Does mode of delivery affect campaign awareness among MSM? (3) Are HIV prevention mass media campaigns acceptable to MSM? (RQ4) What influences campaign acceptability to MSM? (RQ5) Are HIV prevention campaigns effective or costeffective in modifying HIV knowledge or attitudes, reducing sexual risk behaviour, and promoting HIV testing and HIV disclosure among MSM, when compared with preintervention or non-exposed participants? (RQ6) In what ways do intervention characteristics appear to influence awareness, acceptability or effectiveness?

Methods
Our exploratory review was informed by PRISMA guidelines (see Additional file 1) [10]. A protocol was not published but a priori methods were used as described below.

Search strategy
The following electronic bibliographic databases were searched (from January 1990 to May 2011): the Cochrane Library, MEDLINE, EMBASE, Psych Info and ISI Web of Science. Two databases of grey literature, OpenGrey and COPAC, were searched in December 2012. A search strategy using thesaurus and non-thesaurus terms as appropriate to each database relating to the concepts of MSM, media and HIV, adapted from the strategy used by Vidanapathirana et al. (see Additional file 2) [8]. Relevant websites were also searched, including the World Health Organization, Centers for Disease Control, Diffusion of Effective Behavioral Interventions and the Joint United Nations Programme on HIV/AIDS. The reference lists of related reviews and included articles were searched for additional citations. Authors of included studies and other experts in the field were contacted by email to identify further studies.

Criteria for selecting studies
Both published and unpublished literature was included. Included reports met the following criteria: Published in English between 1990 and May 2011.

Target population
Any study where MSM were an intervention's target group, irrespective of sexual identity. This included interventions that solely targeted MSM or where interventions also targeted other groups but study results were reported for MSM as a subgroup. Evaluations where it was not possible to disaggregate the intervention's awareness, effectiveness or acceptability amongst MSM from other target groups were excluded. Studies of campaigns targeting health professionals were excluded.

Intervention
Mass media campaigns relating to HIV health promotion that targeted MSM were included. Unpaid for media coverage and interactive media health promotion interventions (such as use of internet chat rooms) were excluded. Interventions that only included small media, such as leaflets, were excluded, but those where mass media were complemented with small media were included. Outcome evaluations of complex programme interventions which included both a mass media component and non-media components were excluded, as were mass media campaigns relating solely to other aspects of sexual health. Laboratory studies which artificially exposed a research sample to an intervention were also excluded.

Comparators
Pre-intervention or non-exposed study participants.

Study design
As explained above, our review sought to map and appraise a variety of study designs. Therefore, in relation to each of our research questions, studies were not excluded on the basis of their design. Instead, our appraisal systematically assessed the potential internal and external validity of studies (see below). Descriptions of mass media campaigns with no form of evaluation and studies limited to piloting or pretesting were excluded.

Outcomes
Evaluations were included which examined at least one of the following outcomes at any post-intervention timepoint: HIV knowledge or attitudes, HIV testing, HIV disclosure and sexual risk behaviours. No limitations were put on length of follow-up.
References identified through our search were downloaded into an Excel (Microsoft 2010) spreadsheet. Titles and abstracts were screened by one reviewer. Full texts were obtained for review when titles and abstracts met our inclusion criteria or when there was any ambiguity about the decision for inclusion. These were screened by one reviewer, using a screening sheet detailing inclusion and exclusion criteria, and checked by a second with no disagreements occurring.

Data extraction and quality assessment
Data on intervention (media employed, aims, theory of change, any initial formative research or piloting, setting, target population) and study (aims, design, sampling, response rates, data collection, analysis) was extracted from studies by one reviewer and checked by another with any disagreements being resolved by discussion. Data extraction forms had been piloted on two studies.
Quality assessment was conducted at the study level using tools developed specifically for this review. Existing Cochrane [11] and TREND [12] (Transparent Reporting of Evaluations with Non-Randomized Designs) could not be used because these focus respectively on RCTs and on non-randomized studies with external control groups, whereas, as discussed above, our evaluations used a range of designs to examine multiple questions with no consensus in the field as to which are the most rigorous. Quantitative studies were assessed in terms of mimimizing confounding, selection and information bias, reverse causality and random error. Qualitative studies were assessed using established criteria [13] addressing sampling, data collection, data analysis, the extent to which the study findings are grounded in the data, whether the study privileges the perspectives of participants, and the breadth and depth of findings. All reports were quality assessed by one reviewer and checked by another with any differences resolved by discussion.

Synthesis of findings
Given the exploratory aims of this review and the lack of homogeneity in study design and aims, measures and interventions, it was not appropriate to undertake metaanalysis and narrative synthesis was instead undertaken, using similar approaches to those used in previous wellconducted reviews.
Given the small number of qualitative studies found in this review and the lack of overlap in the substantive topics addressed, the decision was taken not to attempt a systematic synthesis, for example via meta-ethnography, and instead to limit reporting to presentation of findings and conclusions of the studies on their own terms.
Across all interventions, posters were mainly placed in gay commercial venues and other gay community settings. Other studies described placing posters in other settings, such as around transport hubs [14,15,28].
Formative research was used to inform the development of the campaign concepts, content and materials in five interventions, including: use of focus groups with the target population [14,15,18,[23][24][25][26]28], meetings with health agencies or community members [16,[23][24][25][26], and community field testing [14,22]. This formative work helped ensure that campaign content addressed men's preferences and needs, and as described in the study by Lombardo & Léger, it also helped to adapt an existing intervention to a new geographical setting while maintaining message consistency [15,28].
Most interventions aimed to provide information on HIV prevention strategies and encourage HIV testing (Table 1). While most of the campaigns had moved away from simple 'use a condom' messages, few aimed to provide men with information on negotiating safer sex or disclosing their HIV status to a sexual partner. Furthermore, only two studies described the theory of change underlying the intervention; one drawing on the Health Belief Model [14] and the other on social marketing concepts [15,28].

Study design and quality
The 12 studies from which our 16 reports drew included a variety of designs; none were solely qualitative studies ( Table 2). Three studies used pre-and post-test repeat cross-sectional designs [14,21,22]. Post-test surveys were conducted between 0-5 months after the campaign launch. Two of these studies also examined outcomes in the post-test survey according to individuals' exposure to the campaign [21,22], while the other examined testing before and after the campaign irrespective of individual exposure [14]. A further six studies involved a single post-intervention cross-sectional survey [15][16][17][18][23][24][25][26][27][28]. Two other studies drew on routine data to examine HIV testing before and after the intervention [19,29]. Another study examined an intervention to increase HIV testing in one sexual health clinic compared with two non-randomized control clinics, collecting data on HIV testing retrospectively [9].
Limited information was provided on recruitment and sampling methods. As expected, none of the studies recruited men using random probability sampling. The study by Lombardo & Léger reported that men were recruited at random from convenience samples in 'gay spaces' rather than using representative sampling frames [15,28]. Other studies reported using convenience sampling to identify men in gay venues, such as bars and clubs, or locations where gay men were likely to congregate, such as on streets or in parks [14][15][16]18,22]. Three studies described recruitment via the Internet [16,21,[23][24][25][26], five accessed GUM attendees [9,[17][18][19]29], and three recruited men at gay community events, such as Pride [18,[23][24][25][26][27]. The study by Katzman et al. was the only one to provide information on response rates and reported a low response rate for surveys distributed at gay venues [16]. Thus, the potential for selection bias is strong resulting in over-estimates of coverage and effects. These limitations also suggest the need for caution regarding the external validity of study findings. Information bias is also likely with many of the studies. Men completing questionnaires face-toface with an interviewer may have been more likely to provide socially desirable responses compared to men self-completing questionnaires, thereby inflating estimates of study coverage, acceptability and effects. Several studies relied on participants' own attributions to assess intervention effects [9,21,22], and these are likely to be vulnerable to information bias.
Three studies included a nested qualitative component [16,[23][24][25][26][27]. None of these studies aimed to build theory. One reported methods of recruitment [23][24][25][26]. Men were purposively sampled to ensure, for example, representation of different ages and HIV statuses. None of the studies reported on methods used to analyse data. Nor did the studies provide in-depth quotes and thick description of context or aim to build theory.          LGBT Lesbian, Gay, Bisexual and Transgender. nG = Non-Gaydar and G = Gaydar. A variety of MSM-related websites were used to recruit men for the survey. However in the post-test Gaydar added. Noted there were differences in reported partnership status and number of partners between Gaydar and non-Gaydar recruited participants, so results presented separately. *'Grey' literature.

Study findings
Findings are presented in Table 2.

RQs 1 and 2: Campaign awareness and how this was affected by mode of delivery
Campaign awareness was examined in nine studies [14][15][16][17][18][21][22][23][24][25][26][27][28]. A variety of designs were used to address the question of campaign awareness: four single crosssectional studies [15][16][17]27,28] and five repeat crosssectional studies [14,18,[21][22][23][24][25][26]. Coverage ranged from 3% to 84%, but this is partly an artefact of different methods of elicitation, such as whether prompted or unprompted and the different time-frames used. Recognition of campaign imagery was in every study more prevalent than recall of campaign messages [14,17,[23][24][25][26]28]. Differences were noted in the characteristics of men who reported campaign awareness compared to those who did not in two studies. Sigma Research observed across surveys that campaigns were more likely to be seen by gay rather than bisexual-identified men, those with more male partners and those who had tested HIV positive compared to those who had not [23][24][25][26]. Men under 20 years of age and those over 50 were less likely to recall campaigns compared with men aged 20-50 years, as were those who had not tested for HIV, and those with low educational attainment. Hilliam et al. reported that non-gay identified men were less likely than gay-identified men to report awareness of the campaign when prompted (69% versus 82%, respectively) [21].
Due to heterogeneity of interventions and methods, it was impossible to determine whether or how mode of delivery affected campaign awareness.

RQs 3 and 4 Campaign acceptability and influences on this
Seven studies reported on acceptability or on the attributes that may affect acceptability [15,16,18,[21][22][23][24][25][26][27][28]. These drew on single cross-sectional [15,16,27,28] and pre/post test cross-sectional design [18,22], as well as focus groups [16,[23][24][25][26]. Hilliam et al. reported that in their evaluation of the "HIV Wake-up campaign" men were most likely to agree the campaign had a clear message (around a third of men) but least likely to agree that the campaign was motivating (less than 20% of men) [21]. Four themes relating to increased acceptability were evident from the focus group studies: imagery (such as the use of models representative of the gay community, the benefits of comics for explicit material, and the importance of ensuring imagery and the campaign message complement one another); content (such as ensuring messages are not too complex); tone (such as not being patronising or blaming); and relevance (such as making certain messages are appropriate to the target audience) [16,[23][24][25][26][27]. The included studies looked at overall acceptability of the campaign rather than the acceptability of different modes of delivery.

RQs 5 and 6 Campaign effectiveness and cost-effectiveness, and effect of intervention characteristics on these
Knowledge and attitudes None of the included studies reported on knowledge or attitude outcomes.

HIV testing and HIV status disclosure
Five studies reported on HIV testing [9,16,19,21] or intention to test [22]. Two of these examined campaigns where the primary aim was to encourage HIV testing [9,22]. A repeat cross-sectional study with data on HIV testing behaviour pre-and post-intervention reported that testing increased significantly post-intervention, from 16% to 33% among gay-identified men and from 9% to 38% in non-gay-identified men [21]. Another study found that 38% of HIV-negative men reported that they would test more frequently as a result of the intervention [22]. However, another study reported that only 9.3% of men attending a sexual health clinic identified a campaign to encourage HIV testing as an important factor in their decision to have a test, although HIV testing data are not presented [17]. In the study comparing intervention and control clinics [9], increases in HIV testing were observed among MSM in general in the campaign clinic, but the greatest increases were observed among Black, southern European and young men, images of whom featured prominently in the campaign. The authors report that whereas in the year prior to the campaign only one of the 65 MSM testing reported that they did so as a result of an advertisement, 162/292 did so during the campaign. No similar increases in testing were observed in the two comparison clinics. However, these findings should be interpreted with caution: men in the control clinics were not asked about the reasons for testing; other confounding factors could have affected the observed increases in testing and there may have been contamination across clinics. A limitation of the two studies in sexual health clinics was that the samples only included individuals having an HIV test [9,19].
HIV status disclosure was examined in one crosssectional study, which observed that 12% of men who reported seeing the campaign reported disclosing their HIV status to a partner [16]. However, as there is no comparison group it is difficult to attribute disclosure of HIV status to the campaign.

Sexual behaviour outcomes
Four studies reported sexual behaviour outcomes: unprotected anal sexual intercourse [9], condom use [16,29], number of sexual partners [29], and change in sexual practice [15,28]. Two of the studies were single crosssectional studies asking men about sexual behaviour post-intervention [15,16,28]. The remaining two studies compared pre-and post-intervention measures [9,29], with neither study observing any significant differences.
However, both these studies were retrospective case note reviews and are therefore limited by what information was recorded in the case notes.

Cost-effectiveness
None of the included studies examined cost-effectiveness of the campaigns.
Due to the broad range of interventions, designs and outcomes, it was not possible to examine how intervention characteristics affected outcomes.

Summary of key findings
The studies included examined different aspects of intervention process and diverse outcomes. Key limitations in methodology included unrepresentative samples, information bias and lack of control of confounding. We note particular problems with the cross-sectional studies comparing post-intervention measures of exposure, process and outcomes, in that these were likely to be subject to strong confounding and information bias [15][16][17][18][23][24][25][26][27][28]. Repeat interrupted times series, a design used by three of the included studies [14,21,22], are likely to provide less biased estimates of intervention effects.
Intervention awareness was variable and recall of key messages among exposed men was generally poor. There was some evidence of lower awareness among non-gay than gay-identified men. Campaign acceptability was variable, and there was some evidence that attention to imagery, content, tone and campaign relevance could enhance acceptability. We found little rigorous evidence of significant effects of mass media interventions on MSM. There was some evidence of short-term increases in HIV testing. Our exploratory review found no rigorous evidence of intervention effects on sexual behaviour outcomes and on HIV status disclosure. As none of the studies were conducted beyond six months, it was not possible to assess sustained impact on behavioural outcomes. Changes in knowledge or attitudes or cost-effectiveness of the campaigns were not reported by any of the studies. We found no evidence addressing how intervention characteristics might influence effectiveness.

Limitations
Because of our multiple research questions and the lack of consensus of which designs are most appropriate to examining these, we undertook an exploratory systematic mapping and appraisal of studies of HIV prevention mass media interventions targeting MSM. In the section below, we reflect on the potential for future systematic reviews of mass media interventions to define more focused questions and inclusion criteria for studies. Other limitations should be noted when interpreting our findings. First, most of the published evidence has come from high income countries. Second, only English language papers were included in the review, so literature in other languages would have been missed. Third, although we searched for unpublished literature in multiple ways, those reports we found came mostly from the UK, which may reflect our stronger domestic networks. Unpublished reports which were older or came from other settings are more likely to have been missed, especially perhaps if they reported null or negative findings. Finally, although interventions that only included small media were excluded, some of the multi-media campaigns did include small media, and it is possible that these may have affected outcomes of interest.

Implications for research
We highlight above the particular weakness of the included studies which drew on post-intervention measures of intervention exposure and outcomes in order to assess intervention effects. Even had these studies attempted to control for confounding differences between those reporting exposure to interventions and those not doing so, substantial residual confounding would very likely have remained because of the subtle differences between those recalling and not recalling campaigns. Therefore, we conclude that any future systematic reviews of mass media should focus on interrupted time-series studies examining pre-and post-intervention measures, drawing either on longitudinal data or from repeat cross-sectional data where this involved consistent sampling methods. Although such studies are vulnerable to confounding by secular trends in the outcomes in question, we conclude that this is a less important source of bias. We therefore also recommend that primary evaluations of mass media effects adopt this design. Nonetheless we stress the importance of cross-sectional studies in assessing awareness and a combination of cross-sectional studies and qualitative research in assessing acceptability.
Our review did not include any studies using interactive media such as smart phone applications or website-based risk assessment tools. Very few studies of such media have yet been published. The use of new technologies warrants further investment and research [30].
Evaluation surveys for the most part found that campaigns were acceptable to MSM. Campaigns need to be relevant to the target audience's needs and formative work during development with target (and non-target groups if material might be viewed outside gay venues) was seen as key to ensuring imagery, language, tone and content were acceptable. Piloting and pre-testing of campaigns should be considered a prerequisite to any campaign launch.

Implications for policy
Mass media interventions have the potential to reach large audiences, and their cost is low per individual reached; though it should be noted that most of the costs provided in the studies focused on display and distribution only, and not on staff time and other development costs. We found insufficient evidence to determine whether mass media interventions represent an effective or cost-effective strategy in the prevention of HIV infection amongst MSM. We recommend that further research is required to investigate this, drawing on interrupted time-series designs and focusing on new/interactive media in addition to traditional/static media.

Conclusion
The aims of the studies in our exploratory review were generally focused on behaviour change, such as HIV disclosure or HIV testing, rather than information provision. A previous systematic review concluded that effective behaviour change interventions require a focus on interpersonal skills development rather than merely the provision of knowledge [31]. The strength of mass media interventions is that they may have a small influence, but on a relatively large portion of the target population. They can also signpost more in-depth interventions, such as one-to-one interventions that are better at addressing motivation and skills. They can set the context in which norms can be changed and stigma addressed, but they cannot affect these things in isolation. This may be an argument for focusing mass media interventions on raising awareness and knowledge and delivering them alongside other more in-depth interventions in order to enable behaviour change.