- Research article
- Open Access
Developing a matrix to identify and prioritise research recommendations in HIV Prevention
BMC Public Health volume 11, Article number: 381 (2011)
HIV prevention continues to be problematic in the UK, as it does globally. The UK Department of Health has a strategic direction with greater focus on prevention as part of its World Class Commissioning Programme. There is a need for targeted evidence-based prevention initiatives. This is an exploratory study to develop an evidence mapping tool in the form of a matrix: this will be used to identify important gaps in contemporary HIV prevention evidence relevant to the UK. It has the potential to aid prioritisation in future research.
Categories for prevention and risk groups were developed for HIV prevention in consultation with external experts. These were used as axes on a matrix tool to map evidence. Systematic searches for publications on HIV prevention were undertaken using electronic databases for primary and secondary research undertaken mainly in UK, USA, Canada, Australia and New Zealand, 2006-9. Each publication was screened for inclusion then coded. The risk groups and prevention areas in each paper were counted: several publications addressed multiple risk groups. The counts were exported to the matrix and clearly illustrate the concentrations and gaps of literature in HIV prevention.
716 systematic reviews, randomised control trials and other primary research met the inclusion criteria for HIV prevention. The matrix identified several under researched areas in HIV prevention.
This is the first categorisation system for HIV prevention and the matrix is a novel tool for evidence mapping. Some important yet under-researched areas have been identified in HIV prevention evidence: identifying the undiagnosed population; international adaptation; education; intervention combinations; transgender; sex-workers; heterosexuals and older age groups.
Other research recommendations: develop the classification system further and investigate transferability of the matrix to other prevention areas; evidence syntheses may be appropriate in areas dense with research; have studies with positive findings been translated to practice?
The authors of this study invite research suggestions relating to the evidence gaps identified within remits of Public Health or any appropriate NETSCC programme.
Follow the 'Suggest Research' links from:
http://www.netscc.ac.uk/. Enter - HIVProject - in optional ID for HTA or in first information box for other programmes.
HIV/AIDS persists as a major global health priority with the number of people living with HIV continuing to increase. A report from the Global HIV Prevention Working Group, a panel of leading AIDS experts, warned that prevention efforts are not keeping pace with the gains being made in treating people infected with HIV. Revitalised global action is required for HIV prevention that supports a combination of behavioural, structural, and biomedical approaches.
In the UK 6,630 people were newly diagnosed as HIV-infected in 2009; half were diagnosed late in infection progression; approximately 22,000 are living with HIV but remain undiagnosed (from unlinked anonymous testing). Although new diagnoses appear to have declined slightly in recent years this reflects undiagnosed infection and diagnoses made abroad. Undiagnosed infection presents two major problems: i) unaware onward transmission of HIV to others, ii) complications in presentation and treatment when late stage HIV is detected.
The scale and nature of sexual ill health and inequalities in England are of concern to policy makers and consideration is currently being given to key action points in the national Sexual Health Strategy expected in 2011. The UK Department of Health has set the strategic direction for delivering healthcare with a greater focus on prevention as part of its World Class Commissioning Programme. This includes recognition of the long term cost effectiveness of prevention strategies and the need to prioritise prevention to ensure viability of the NHS in the long term. There is potential for the methods outlined in this study to contribute to this aim through focused research directions.
Evidence mapping can make a positive contribution to policy agenda, with several research and policy gaps being fed into existing prioritisation channels: this was a concluding point from the Specialty Mapping Pilot by J. Shepherd in 2007. It is one of the few published examples of innovative methodology to identify and prioritise topics for HTA and it concludes by recommending specialty mapping in other topic areas with on-going evaluation: these points were the catalyst for our study.
This study began on the premise that HIV prevention is an area in need of novel evidence-based interventions and that important yet under-researched areas within HIV prevention exist: these could be identified through thematic categorisation of the many possible combinations of risk groups and prevention related activities. We therefore devised a categorisation system for HIV prevention and piloted a matrix tool to identify gaps amongst the many contemporary studies on HIV prevention from countries with possible relation to HIV in the UK.
A literature scoping exercise was undertaken to identify existing methodologies for conducting this work. The closest appropriate were the Global Evidence Mapping Initiative and the Speciality Mapping Pilot . These were discarded, however, because both were more suited to treatment care pathways than prevention. A new matrix system for mapping evidence was developed to collate publications into groups and combinations of prevention related activities, risk groups and study types. This was not an evidence synthesis and restricted resources did not allow for quality appraisal of the studies beyond mapping according to hierarchy of evidence.
Development of the matrix
Four external experts were consulted and participated in the proposed classifications. These classifications represent important structural, contextual, behavioural, biomedical, service approaches, risk groups and combinations of these (see Additional file 1). There are important areas of research such as epidemiology that require investigation before interventions can be designed. A pilot was conducted on 100 publications to assess the robustness of the matrix and adjustments made accordingly.
A search was made of the NIHR HTA programme Access databases (PROMIS) using the key term 'HIV' from inception in 1995 to March 2009. All topics identified and screened for being HIV prevention specific were eligible for inclusion. This longer time period was searched to provide additional internal audit, but research prior to 2006 was later excluded to match the time period of this study. A systematic search of major electronic databases including MEDLINE, MEDLINE In-Process, EMBASE, and Health Management Information Consortium (all Ovid) and the Cochrane Library were then searched using a carefully designed and tested search strategy (see Additional file 2). Primary research results were geographically filtered to studies undertaken in the UK, USA, Canada, Australia and New Zealand. Filtering by study location is liable to indexer error and bias. We were primarily interested in HIV prevention in the UK but USA publications were included as their vast research programme is a source of evidence for many other western countries. Canada, Australia, and New Zealand were included as other English speaking, developed countries with similar HIV epidemic and health service issues. To reduce publication bias we also searched a grey literature database for NHS reports.
Published studies were searched from 2006 to July 2009 as HIV is a rapidly changing area of health and contemporary evidence was deemed more useful in this exploration of the matrix. Due to the time and resource constraints the searches were limited to English language only, which is considered an acceptable limitation under such circumstances[10, 11].
Any study where the stated aim is to investigate HIV prevention through epidemiology, policy, methods or various interventions; studies which aim to reduce infectivity of HIV positive people by reducing viral load (VL) through treatment adherence and reducing treatment resistance; studies which investigate behavioural and social factors stated to relate to risk of HIV infection; English speaking countries only; studies from UK, USA, Canada, Australia and New Zealand; publication dates 2006-2009 inclusive; systematic reviews, randomised controlled trials and other primary research of high quality (hierarchy of evidence) - case-controls, surveys, cohorts.
Studies which explore merits of HIV treatments but are not stated to be concerned with reducing viral load (VL)/infectivity to others e.g. side effects; depression/stress studies that do not mention links to risk of transmission or reduction of VL; studies that relate to HIV but are not stated to be concerned with risk of transmission e.g. relationships with children; primary research of low quality (hierarchy of evidence) - case studies, expert opinion, anecdotal evidence, narrative reviews.
Search results, coding and quality assurance
Search results were downloaded to three electronic Reference Manager (version 11) databases according to study types: systematic reviews; randomised controlled trials; other primary research (prospective/retrospective cohort studies, surveys, case-controls, but not case series). Each publication was screened for study type, date of publication, geographical area and if actually HIV prevention (see Inc/Exc criteria above). Included studies were screened against definitions (Additional file 1) and coded/counted accordingly. Many publications focused on more than one risk group, therefore achieved more counts e.g. A behavioural intervention for young, socially excluded drug-users counted 1 in each of the risk group cells under behaviour heading on the matrix = 3 counts. For verification purposes, search findings were checked against publications identified by Mimas Zetoc Alert Service for HIV prevention articles. The project lead was responsible for all coding of records, thereby not requiring inter-observer variation checks, however, quality assurance was conducted by a second team member taking random samples from each database to validate against our criteria.
Population of the matrix
Coded records were exported from Reference Manager to the matrix in an Excel spreadsheet which contained formulas for placing and counting the codes in corresponding cells within the matrix. Each cell was subdivided by study type, each column and row was totalled and each area with no research was left blank for easy visibility.
Publication search results, coding and quality assurance
From 1995 to March 2009 NIHR, HTA received 80 suggestions and proposals relating to HIV prevention. Out of the 80 suggestions five were prioritised and research subsequently funded. Publications were evident for two of the five studies although one of these was excluded due to being outside the inclusion date range 2006-9 (the other studies are still ongoing). The one remaining included study was a systematic review on post-exposure prophylaxis (PEP).
From electronic database searches we identified 1648 publications across the three research design groups. 40 were excluded in the initial screening for duplications. Through Mimas Zetoc Alert Service for HIV prevention articles two additional articles were found and added to our databases. A total of 1610 international publications were suitable for further analysis (Table 1). These studies were rigorously screened, coded and counted against inclusion criteria. Many publications focused on more than one risk group and were counted again. A second team member conducted quality assurance on coding: random samples from each database (n = 206) were checked: n = 8 errors were found (3.9%): two coded incorrectly and six required an additional code to one already given. 716 studies have 1 risk group; 127 studies - 2 risk groups; 25 studies - 3 risk groups = Total counts 868 (see full bibliography of included studies in Additional File 3).
Population of the matrix
After verification, the scores of the included studies were exported to the appropriate cells in the matrix. First impressions from the high numbers of studies identified suggest that HIV prevention is generally a heavily researched area, however, the results shown in the matrix draw attention to some unexpected exclusions within this field. The dense, sparse and empty areas of research are visually evident within HIV prevention activities, risk groups and study types - See the matrix in Figure 1 and areas for prioritisation below.
This method and tool could easily and rapidly be replicated in other prevention areas such as obesity by: a) defining headings for the matrix, b) search literature, c) code publications and export to matrix.
Areas for prioritisation from the matrix findings
The matrix has highlighted several important areas within HIV prevention research that represent potential for novel and innovative research of interest to practitioners, commissioners, policy makers and researchers alike. We detail below those evidence gaps we believe would be most useful for future research in HIV prevention:
Sero-sorting: identifying the undiagnosed population
The Health Protection Agency (HPA) 2010 report shows over a quarter of people who are HIV+ve are not aware of their status. Our analysis showed only four papers[13–16] investigating this area, all of which examine sero-status awareness and undiagnosed infection amongst gay men or undefined populations. A difficulty with sero-sorting is possible overlap with testing issues, which, for the purposes of this study, have a different focus - Additional file 1.
the majority of HIV prevention research originates from USA yet the matrix identified just three publications[17–19] on research to test possible cultural translation and adaptation for use in other countries. Two of these studies examined the difficulties of adaptation, but the third adapted a successful intervention from USA in London and has since been adopted UK wide. Gay men and young people were the only risk groups addressed here.
We searched for studies investigating the effects of interventions such as prosecutions and policies regarding confidentiality, mortgages, travel restrictions and ethics on HIV prevention. Three publications[20–22] were identified addressing sex workers, ethnic minorities and undefined risk groups. Admittedly, this is an area where study design may present some difficulties. All identified studies examined the impact of public policy initiatives.
Combination packages of interventions
Limited HIV prevention resources could be optimised through synergy. We identified ten publications addressing this area. The majority of these were systematic reviews that analysed more than one intervention. Only two[23, 24] of these studies aimed to examine the value of combining levels of approaches.
Social/population level factors
Relative to other areas of prevention social factors are revealed to be researched in moderation; 25 studies were identified. These studies examine factors such as gender, race, housing, poverty; only two[25, 26] of these investigate stigma and discrimination from the health provider perspective or the epidemic as a whole.
The matrix also revealed fewer than anticipated studies in this area of prevention. The matrix found 28 counts in this area, the majority of which focused on impact of traditional school education, leaflets and videos: small group or individual exposure. Surprisingly, there were only two publications[27, 28] on the effectiveness of HIV mass-media campaigns.
Interestingly, the matrix has illustrated some least targeted but potentially emerging risk groups. These include older age groups, transgender, sex workers, male, heterosexuals, other ethnic (besides black/African), socially excluded and other vulnerable risk groups. Our findings indicate that risk groups such as HIV positive, drug and alcohol users, Men who have sex with men and young people are the most targeted groups for study sample populations. However, examining the quality of these studies went beyond the scope of the current research.
To our knowledge this is the first study to identify research gaps in HIV prevention using this novel method. In order to develop the matrix, we created a classification of prevention related activities and risk groups; a novel building block of the study as called for by Akers in 2003. The development of the matrix has provided a tool which we believe has potential transferability for scoping other disease prevention areas such as obesity or type II diabetes; which we hope will be investigated further. We have identified six key under-researched prevention areas (Table 2).
There was no facility within the search strategy or Reference Manager to identify multiple publications from one study. A manual search of a quarter of the database found 11 publications from 5 studies. As there is no systematic way to identify multiplications or adjust the matrix cells with certainty we could extrapolate from this quarter sample and estimate that 44 papers (out of 717 included) will come from 20 studies. However, in our conclusions we concentrate on the least populated areas of the matrix (Table 2) and found no multiple publications amongst them.
Publications that focused on more than one risk group were problematic. By consensus the project team agreed to repeat the count according to the risk groups addressed. This appeared the only way to fairly represent groups studied; however, this can cause confusion when the counts of risk groups and prevention areas do not match the numbers of publications identified. This is an area for potential improvement.
The limited remit of this study did not enable us to critically appraise beyond the hierarchy of evidence. In heavily researched areas like behaviour change, the density of evidence does not necessarily equate to high quality, robust research or provide evidence that translates to effective prevention. However, in the USA the Centre for Disease Control have already set up a project to synthesise evidence in relation to behavioural interventions: The Prevention Synthesis Project.
HIV prevention in the UK remains problematic; effectiveness of current prevention efforts appears limited; novel and innovative, evidence-based approaches are needed in HIV prevention. HIV prevention remains one of the great challenges for sexual health in the UK. This study offers a method for identifying possible future directions in research. This, in turn, may lead to a more complete understanding of effective prevention approaches.
A matrix tool and classification system has been developed to identify important research gaps in HIV prevention; both may be useful in other health areas.
Several important gaps in English language HIV prevention research since 2006 were identified by the matrix: identifying and understanding the undiagnosed population; International adaptation of research; education; intervention combinations. The key under-researched risk groups were transgender, sex-workers, heterosexuals and older age groups.
Three additional future research recommendations have been identified during this study: investigate further use of the classification system we have developed and transferability of the matrix to other health prevention areas; evidence syntheses may be appropriate in areas dense with research, if not already being carried out by CDC Prevention Research Synthesis Project (Behaviour); have studies with positive findings have been translated to practice and how many recommending future research have been follow-up with funded studies?
The authors of this study invite research suggestions relating to the evidence gaps identified within remits of Public Health or any appropriate NETSCC programme.
Follow the 'Suggest' research links from:
http://www.netscc.ac.uk/ Enter - HIVProject - in optional ID for HTA or in first information box for other programmes.
National Institute for Health Research
NIHR: Evaluation, Trials and Studies Coordinating Centre (NETSCC)
Health Technology Assessment
Human Immunodeficiency virus
randomised controlled trial
Health Protection Agency.
UNAIDS Epidemic Update 2009. [http://www.unaids.org/en/KnowledgeCentre/HIVData/EpiUpdate/EpiUpdArchive/2009/default.asp]
Hargreaves S: More emphasis needed on HIV prevention, warn experts. The Lancet Infectious Diseases. 2007, 7: 508-
Merson MH, O'Malley J, Serwadda D, Apisuk C: The history and challenge of HIV prevention.[see comment]. [Review] [88 refs]. Lancet. 2008, 372: 475-488. 10.1016/S0140-6736(08)60884-3.
HIV in the United Kingdom: 2010 Report. [http://www.hpa.org.uk/Publications/InfectiousDiseases/HIVAndSTIs/1011HIVUK2010Report/]
HIV in the United Kingdom: 2010 Report. [http://www.hpa.org.uk/web/HPAweb&HPAwebStandard/HPAweb_C/1287145264558]
Equality Impact Assessment for National Sexual Health Policy. [http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_111227]
World Class Commissioning. [http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_080952.pdf]
Shepherd J, Briggs J, Payne L, Packer C, Kerridge L, shton-Key M: Setting the future policy agenda for health technology assessment: A specialty mapping approach. Int J Technol Assess Health Care. 2007, 23 (4): 405-413.
The Global Evidence Mapping Initiative. [http://www.evidencemap.org/about]
English-Language Restriction When Conducting Systematic Review-based Meta-analyses: Systematic Review of Published Studies.
CRD's guidance for undertaking reviews in health care.
National Coordinating Centre for Health Technology Assessment: Non-occupational post-exposure prophylaxis for HIV: a systematic review (DARE structured abstract). Health Technology Assessment. 2009, 13:
Dodds JP, Johnson AM, Parry JV, Mercey DE: A tale of three cities: persisting high HIV prevalence, risk behaviour and undiagnosed infection in community samples of men who have sex with men.[erratum appears in Sex Transm Infect. Sex Transm Infect. 2007, 83: 392-396. 10.1136/sti.2006.021782. 2007 Oct;83(6):500]
Holtgrave DR, Pinkerton SD: Can increasing awareness of HIV seropositivity reduce infections by 50% in the United States?. J Acquir Immune Defic Syndr. 2007, 44: 360-363. 10.1097/QAI.0b013e31802ea4dd.
Pinkerton SD, Holtgrave DR, Galletly CL: Infections prevented by increasing HIV serostatus awareness in the United States, 2001 to 2004. J Acquir Immune Defic Syndr. 2008, 47: 354-357. 10.1097/QAI.0b013e318160d57e.
Williamson LM, Hart GJ: HIV prevalence and undiagnosed infection among a community sample of gay men in Scotland. Journal of Acquired Immune Deficiency Syndromes: JAIDS. 2007, 45: 224-230. 10.1097/QAI.0b013e318058a01e.
Bonell C, Strange V, Allen E, Barnett Page E: HIV prevention outreach in commercial gay venues in large cities: evaluation findings from London. Health Education Research. 2006, 21: 452-464.
Morrison DM, Hoppe MJ, Gillmore MR, Kluver C, Higa D, Wells EA: Replicating an intervention: the tension between fidelity and adaptation. AIDS Educ Prev. 2009, 21: 128-140. 10.1521/aeap.2009.21.2.128.
Zometa CS, Dedrick R, Knox MD, Westhoff W, Siri RS, Debaldo A: Translation, cross-cultural adaptation and validation of an HIV/AIDS knowledge and attitudinal instrument. AIDS Education & Prevention. 2007, 19: 231-244. 10.1521/aeap.2007.19.3.231.
Eshel A, Moore A, Mishra M, Wooster J, Toledo C, Uhl G, Aguero LW: Community stakeholders' perspectives on the impact of the minority AIDS initiative in strengthening HIV prevention capacity in four communities. Ethn Health. 2008, 13: 39-54. 10.1080/13557850701803155.
Shannon K, Strathdee SA, Shoveller J, Rusch M, Kerr T, Tyndall MW: Structural and environmental barriers to condom use negotiation with clients among female sex workers: implications for HIV-prevention strategies and policy. Am J Public Health. 2009, 99: 659-665. 10.2105/AJPH.2007.129858.
Hornberger J, Holodniy M, Robertus K, Winnike M, Gibson E, Verhulst E: A systematic review of cost-utility analyses in HIV/AIDS: implications for public policy. [Review] [128 refs]. Med Decis Making. 2007, 27: 789-821. 10.1177/0272989X07306112.
Herbst JH, Beeker C, Mathew A, McNally T, Passin WF, Kay LS, Crepaz N, Lyles CM, Briss P, Chattopadhyay S, et al: The effectiveness of individual-, group-, and community-level HIV behavioral risk-reduction interventions for adult men who have sex with men: a systematic review.[see comment]. [Review] [165 refs]. Am J Prev Med. 2007, 32: S38-S67.
Slesnick N, Kang MJ: The impact of an integrated treatment on HIV risk behavior among homeless youth: a randomized controlled trial. J Behav Med. 2008, 31: 45-59. 10.1007/s10865-007-9132-5.
Kinsler JJ, Wong MD, Sayles JN, Davis C, Cunningham WE: The effect of perceived stigma from a health care provider on access to care among a low-income HIV-positive population. AIDS Patient Care STDS. 2007, 21: 584-592. 10.1089/apc.2006.0202.
Mahajan AP, Sayles JN, Patel VA, Remien RH, Sawires SR, Ortiz DJ, Szekeres G, Coates TJ: Stigma in the HIV/AIDS epidemic: a review of the literature and recommendations for the way forward. [Review] [89 refs]. AIDS. 2008, 22 (Suppl 2): S67-S79. 10.1097/01.aids.0000327439.20914.33.
Bertrand JT, O'Reilly K, Denison J, Anhang R, Sweat M: Systematic review of the effectiveness of mass communication programs to change HIV/AIDS-related behaviours in developing countries. Health Educ Res. 2006, 21: 567-597.
Vidanapathirana J, Abramson MJ, Forbes A, Fairley C: Mass media interventions for promoting HIV testing: Cochrane systematic review. [Review] [1 refs]. Int J Epidemiol. 2006, 35: 233-234.
Akers TA, Hervey WG: Why classification for HIV/AIDS prevention interventions?. JANAC. 2003, 14: 17-20.
Gupta GR, Parkhurst JO, Ogden JA, Aggleton P, Mahal A: Structural approaches to HIV prevention. [Review] [76 refs]. Lancet. 2008, 372: 764-775. 10.1016/S0140-6736(08)60887-9.
HIV/AIDS Prevention Research Synthesis Project (Behaviour). [http://www.cdc.gov/hiv/topics/research/prs/about.htm]
Next Steps for Improving Sexual Health. [http://www.dh.gov.uk/en/Publichealth/Healthimprovement/Sexualhealth/Sexualhealthgeneralinformation/DH_4079794]
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/11/381/prepub
We would like to thank the additional following external experts for their valuable help with this project: Dr. Jonathan Shepherd, Dr. Mike Youle, Mr. Glenn Turner and Dr. Colette Smith.
All files can also be found at: http://www.netscc.ac.uk/supporting_research/research_on_research_publications.asp
This research was carried out by the NIHR Evaluations Trials and Studies Coordinating Centre special interest Research-on-Research group. No additional funding was sought for this research.
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Department of Health or NETSCC.
Data sharing: No additional data available
Licence for publication statement: The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non exclusive for government employees) on a worldwide basis (if accepted) to be published in the BioMed Central Public Health Journal and sub-licences such use and exploit all subsidiary rights.
The authors declare that they have no competing interests.
SA wrote the first draft of the paper and led the writing process. AP designed the search strategy and performed the publication searches. SA, BM and AJ analysed the data. BM contributed to quality assurance of data coding. All authors contributed to study design and provided input into the revisions of the paper. All authors were involved in the research design or analysis and approval of final draft submitted.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Anstee, S., Price, A., Young, A. et al. Developing a matrix to identify and prioritise research recommendations in HIV Prevention. BMC Public Health 11, 381 (2011). https://doi.org/10.1186/1471-2458-11-381
- Risk Group
- Prevention Area
- International Adaptation
- Matrix Tool
- Undiagnosed Infection