Public health triangulation: approach and application to synthesizing data to understand national and local HIV epidemics
© Rutherford et al; licensee BioMed Central Ltd. 2010
Received: 21 March 2010
Accepted: 29 July 2010
Published: 29 July 2010
Public health triangulation is a process for reviewing, synthesising and interpreting secondary data from multiple sources that bear on the same question to make public health decisions. It can be used to understand the dynamics of HIV transmission and to measure the impact of public health programs. While traditional intervention research and metaanalysis would be ideal sources of information for public health decision making, they are infrequently available, and often decisions can be based only on surveillance and survey data.
The process involves examination of a wide variety of data sources and both biological, behavioral and program data and seeks input from stakeholders to formulate meaningful public health questions. Finally and most importantly, it uses the results to inform public health decision-making. There are 12 discrete steps in the triangulation process, which included identification and assessment of key questions, identification of data sources, refining questions, gathering data and reports, assessing the quality of those data and reports, formulating hypotheses to explain trends in the data, corroborating or refining working hypotheses, drawing conclusions, communicating results and recommendations and taking public health action.
Triangulation can be limited by the quality of the original data, the potentials for ecological fallacy and "data dredging" and reproducibility of results.
Nonetheless, we believe that public health triangulation allows for the interpretation of data sets that cannot be analyzed using meta-analysis and can be a helpful adjunct to surveillance, to formal public health intervention research and to monitoring and evaluation, which in turn lead to improved national strategic planning and resource allocation.
Public health practitioners are frequently faced with having to make programmatic decisions with less than perfect data. It is seldom the case that a high-quality study exists that can provide answers to everyday public health questions directly, and often we must rely upon imperfect surveillance data to guide policy and programmatic decisions. This has been particularly the case for HIV, where the urgency of the epidemic coupled with enormously scaled-up international resources demand rapid responses in planning and targeting prevention and care programs. Further, the last several years have seen a massive increase in not only HIV surveillance data but also in programmatic data, quantitative and qualitative research studies, and local and national experience and expertise. While the volume of information has increased exponentially, its use and analysis to inform program, planning and policy have been lagging. Diverse sources of data are rarely presented together, and gathering, synthesizing and interpreting them has become increasingly challenging.
"Triangulation" is a term that can broadly refer to an approach to synthesizing multiple, diverse sources of data at the level of interpretation. However, the word itself has come to mean different things in different fields. Social scientists have used the term "triangulation" since the late 1960 s . By triangulation they mean examining multiple data sources to validate results, increase credibility and gain a more detailed understanding of findings [2, 3]. The term has been used to refer to methods for establishing both internal and external validity by decreasing the uncertainty of a single measurement by making multiple observations . Denzin describes four types of triangulation: data triangulation, in which data gathered through different samples and at different times are compared; investigator triangulation, in which more than one investigator examines the same question and results are compared; theory triangulation, in which different theoretical constructs are applied to the same observed data; and method triangulation, in which phenomena are examined using different methods . The term triangulation has also been used in nursing research to refer to mixed qualitative and quantitative methods to explain complex phenomena [6–8], akin to Denzin's method triangulation. In these contexts, "triangulation" often implies that the approach was created at the design stage, where the investigator could exert control over the study methods and measures. Triangulation is similar to evidence mapping [9, 10], and realist synthesis [11, 12]. These, however, tend to be more forward-looking and less concerned about explaining what has transpired in the past, creating the present day situation. It is also similar to narrative reviews but tends to have a broader focus than individual interventions . Triangulation, as used in the context of public health, is probably most closely related to critical interpretive synthesis, which has as its focus the generation of theory to interpret observations and is not nearly as methodologically constrained as meta-analysis . However, as discussed below, triangulation extends the construct of critical interpretive synthesis by addressing not only how we got to where we are but also what we do next.
More recently the term has been used in the context of public health to refer to the process of reviewing and interpreting secondary data from multiple data sets that bear on the same question to make public health decisions, combining elements of both data triangulation and method triangulation [15, 16]. The work of Stoneburner and Low-Beer in Uganda offers an early example of this approach to triangulation . Their basic question was whether or not the declining trends in HIV prevalence seen in women attending antenatal clinics in Kampala and elsewhere in the early to mid 1990 s were the result of declines in HIV incidence or the result of mortality. The scope of the question and the timeframe encompassed did not allow for the design of a single prospective study that could answer this question directly and definitively. The changes during this period were substantial with HIV prevalence falling from 21.1% to 9.8% between 1991 and 1998 and to 6.4% in 2001  based on data from antenatal clinic sites. They were able to confirm similar trends in other national datasets, such as a decline in HIV prevalence in male army recruits and blood donors and declines in directly measured HIV incidence in large cohort studies in the country . They also identified a series of data sets that bore on variables in the chain of events that led to prevalent HIV infection in women, including age at sexual debut, risk factors for exposure to HIV (men's and women's numbers of partners, communication about avoidance of risk), risk factors for transmission (use of condoms), incident HIV infection and mortality. They were able to show that an increase in age of sexual debut and decreasing numbers of sexual partners, which would have led to contraction of sexual networks, temporally preceded the decline in HIV prevalence . Moreover, data from neighboring countries with HIV epidemics in a similar stage as Uganda did not show comparable declines in early sexual debut and number of sex partners nor subsequent declines in HIV prevalence, further suggesting that behavior change rather than mortality was the cause of the decline in HIV prevalence seen in Uganda . Additionally, prevention programs at the time emphasized partner reduction or "zero grazing". The success of Uganda was heralded as evidence that national level responses could produce sweeping impact on the HIV epidemic. In this example, we highlight the success of the triangulation approach in assembling and interpreting diverse existing data sources to provide evidence in support of the underlying cause of the national epidemic trend. There are several other examples where public health triangulation has been used to understand trends, for instance, from Cambodia, Thailand and the United States [21–25], and the 5-year evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria  is using not dissimilar methods
In this paper we propose a standard approach to public health triangulation and suggest scenarios in which it can enhance understanding of the national and local HIV epidemics and prove useful in programmatic decision-making. We define public health triangulation as the process of reviewing and interpreting existing data and trends in those data from multiple data sources that bear on different facets of a broad public health question in order to identify factors that underlie the observed data and to assist with public health decision-making and actions.
Approach to Public Health Triangulation
Use of existing data; on occasion very limited new data abstraction or cross-matching between data sets;
Synthesis of data qualitatively, similar to what is done in a narrative review, in contrast to combining data sets quantitatively for statistical analysis;
Inclusion of diverse data sources, such as surveillance, research, programmatic and expert opinion; both quantitative and qualitative data; and both biological and behavioral measures;
Input from stakeholders for the formulation of a public health question, data identification and assessment of data, interpretation and dissemination of results;
Using results to inform public health decision-making
While the main goal of triangulation is to inform public health decisions, triangulation can also be used to assess the external validity of observed trends as well as to examine the effectiveness of widely disseminated interventions on multiple outcomes, such as knowledge, attitudes, behaviors and actual disease incidence, at the population level. Whereas traditional intervention research and systematic reviews seek to answer pre-formed hypotheses, triangulation seeks to strengthen interpretations and improve decisions based on the available evidence. Triangulation does not formally demonstrate causality in the same manner as a purposefully designed randomized controlled trial but rather offers a rational explanation or interpretation of the data at hand.
How public health triangulation differs from conventional epidemiologic analysis
Public health triangulation analysis
Conventional epidemiologic analysis
Emphasis on 'best possible' existing data
Emphasis on data of highest scientific rigor
Focus on plausibility as basis for conclusions (with or without statistics)
Focus on statistics as basis for conclusions
Focus on external validity: "Can observed effects be generalized to the larger population?"
Focus on internal validity: "Did A cause B in our study?"
Based on inter-connected pieces of the same situation
Based on independent samples
Goal: public health decision-making
Goal: increasing scientific knowledge
Data triangulation and synthesis activities in general are key components for monitoring HIV epidemics and should be incorporated into routine monitoring and evaluation systems. Most countries already conduct these activities at some level; an example is developing an epidemiological profile as the first part of a national prevention plan. In addition, these activities are part and parcel of several focused efforts including the World Health Organization's (WHO) Second-Generation Surveillance , the Policy Project's A-Squared: Analysis & Advocacy , UNAIDS and World Bank's Modes of Transmission Study , and UNAIDS' "Know Your Epidemic" . All these activities tap into similar data sources and share some objectives; however, they are more heavily focused on understanding HIV incidence and prevalence, use specific arrays of data and often involve modeling. In contrast, public health triangulation uses a different approach and supports the added dimensions of stakeholder-driven processes for generating key questions, identifying relevant existing data sources, formulating hypotheses and interpreting and using data.
The public health triangulation process
The public health triangulation process.
1. Identify key questions
2. Ensure question(s) are important, actionable, answerable and appropriate for triangulation
3. Identify data sources and gather background information
4. Refine the questions
5. Gather data/reports
6. Assess data reliability and make observations from each data set
7. Note trends across data sets and hypothesize
8. Check (corroborate, refute, modify) hypotheses
9. If necessary, identify additional data and return to Step 5
10. Summarize findings and draw conclusions
11. Communicate results and recommendations
12. Outline next steps for public health action
Framing a question
Criteria for a question answerable by public health triangulation
Public health importance
Is the question being asked of sufficient public health importance to justify the investment of human resources and funding? [e.g., should a question regarding the impact of rare injection drug use in an African country be pursued or one focused on much more prevalent alcohol use?]
Will answering the question being asked lead to the initiation of a public health action? Will the risk factors we identify be modifiable and amendable to public health interventions?
Are there data available that have asked the right questions and provide answers on the different steps in the sequence of events that leads to a public health outcome?
Can the question be answered with conventional research methods or with a single available data set? Is a proposed intervention sufficiently new and unique that it should be evaluated by a different methodology?
Are there sufficient human resources and funding available to gather and analyze the data? Unless sufficient resources are available, searching for data and conducting the multiple levels of analysis needed for triangulation may be inadequate
Conceptually, the selection of a public health triangulation question is two-step: first focusing on whether the question is important and actionable (broad policy considerations) and second on the logistical considerations. Logistical considerations can often be addressed with more detailed fieldwork, whereas if the broad policy considerations are not met, the field effort may not be worthwhile. The process should guide stakeholders towards questions that are feasible, actionable and most amenable to public health triangulation. Note that the focus of these questions should be on identifying modifiable risk factors that are amenable to public health interventions rather than on non-modifiable risk factors, such as age or sex. Frequently, this helps prioritize questions to be examined. For example in Malawi, stakeholders identified 33 questions initially; two were chosen after extensive discussion:
▪ Has HIV prevalence (incidence) increased, decreased or remained the same in Malawi from 2000 to 2005 and has this change been due to a modifiable risk factor?
▪ What was the reach and intensity of HIV prevention programs in Malawi from 2000 to 2005?
These two key questions were eventually selected based on how actionable the answers might be (scored as high, medium and low) and what data sources were available. In contrast, for example, one proposed question, "What is the relationship between alcohol and risk behavior?" received lower priority. Although alcohol use is important and potentially actionable, clear interventions to reduce alcohol-related transmission have not been successful in the context of generalized epidemics. Moreover, there were only a few questions from prior Demographic and Health Surveys that could have provided data.
Understanding the sequence of events that leads to an outcome
In order to posit a causal association between a specific modifiable variable (e.g. multiple partners) and an outcome (e.g., HIV incidence), among other criteria, changes in trends for the variable should temporally precede changes in the outcome. This is a necessary but not sufficient criterion for assessing causality. The most useful data for triangulation are therefore those that describe trends in predictor and outcome variables that can logically show the sequence of events. For instance, a variety of behaviors and biological events, such as early age at first intercourse, multiple concurrent sexual partners, having ever paid for sex, not using condoms, having had a sexually transmitted infection recently and not being circumcised, are all risk factors for either exposure to HIV or, if exposed, sexual transmission of HIV. Changes in these variables should therefore logically precede or coincide with changes in HIV prevalence or incidence. Moreover, these examples are all modifiable risk factors, as opposed to non-modifiable risk factors such as age or sex. For example, in Uganda declines in the numbers of non-regular partners among men temporally preceded declines in HIV prevalence among women seeking antenatal care . Additionally, other variables may modify these risks and need to be considered on the causal pathway. An example might be factors that protect against exposure, such as an infected sexual partner's low viral load, or if exposure occurs, against transmission, such as circumcision.
As with conventional systematic reviews and meta-analysis, searching for evidence is the most important task in public health triangulation. Once identified, all relevant data and information are initially considered. Typically, data originate from public health and household surveillance, national censuses, public health programs and smaller epidemiologic research studies that often appear only in abstract form or are presented at regional or national meetings. These data are identified through conventional data searches and with the assistance of ministries of health, non-governmental organizations and local academic institutions, which are typically represented among the stakeholders, and in our experience, which has primarily focused on HIV/AIDS in low- and middle-income countries, there are large amounts of data available. Both quantitative and qualitative data are useful, and understanding of the local context in which one is trying to answer a question is often enriched with qualitative data rather than relying completely on quantitative data. Moreover, local knowledge and expert opinion are particularly important in understanding potential relationships between predictor and outcome variables. Identifying and accessing the relevant data are requisite steps in being able to use these methods.
Assessing data quality and reliability
Some data sources are better than others in the sense that those generating the data have taken steps to minimize bias, control for confounding and maximize the likelihood of generalizability and external validity. For example, HIV surveillance data are often a core data source for triangulation, so sampling frameworks, selection of participants, underreporting and non-response bias are important considerations when assessing representativeness. While we can use formal quality criteria for observational and experimental studies [32, 33], we are more frequently faced with assessing quality of survey data for which formal evaluation criteria have not been published. In lieu of formal quality criteria for surveys and surveillance studies, the following questions, largely bearing on the representativeness of the sample and the generalizability of the results, can be used to assess external validity:
From where was the sample selected and why?
Who was selected and why?
How were they selected and why?
Was the sample size justified?
Is it clear why some participants chose not to take part?
Criteria for ranking data sources.
• Uniform data collection and reporting tool used?
• Frequency and timeliness of reporting
• Adherence to a standard operating procedure for data management?
• Data coverage (Number of sites included in the reporting system/total number of sites offering services)
• Sample size (patient, client, or product (i.e. condoms))
Surveillance data (ANC, DHS, BSS, etc.)
• Representativeness of sample for the target population (probability based?)
• Implementation (implemented according to protocol)
• Strength of the measures (biomarkers, self report, detailed behavioral indicators, knowledge)
• Analysis (appropriate, complete, methods detailed)
• Frequency and timeliness of data collection and reporting
• Consistency of sites/locations and populations measured over time
• Sample size
• Response rate
Special studies (qualitative)
• Sampling strategy explained in detail
• Interview/observation methods described in detail
• Implementation (methods implemented according to protocol)
• Analysis (appropriate, complete, methods detailed)
• Sample size
Special studies (quantitative)
• Representativeness of sample (probability based?)
• Implementation (methods implemented according to protocol)
• Strength of the measures (biomarkers, detailed behavioral indicators, knowledge)
• Analysis (appropriate, complete, methods detailed)
• Sample size
• Response rate
Many of the data sets we have dealt with address trends over time. Thus, consistency of sampling and data collection methods and sufficient sample size to detect significant trends are important as well. For instance, an HIV sentinel survey of prepartum women attending antenatal clinics should ideally use the same clinics from year to year or at least present data in a way that we can ascertain trends in the clinics that have been surveyed longest and most consistently.
Formulating and assessing hypotheses
At its core public health triangulation seeks to confirm or refute explanatory hypotheses with available data. This process starts with the key questions and review of data by person, place and time for each data set using tables, figures and maps. We then examine trends in key indicators (both predictor and outcome variables) to see if there is a logical temporal and spatial relationship among trends in one or more indicators (predictor variables) and outcomes and across multiple data sets. Ideally, we can corroborate the same or similar indicator trends in different data sets (e.g., a decline in HIV prevalence in women attending ANC, a decline in HIV prevalence in military recruits, a decrease in HIV prevalence in two Demographic and Health Survey with HIV testing (DHS+ surveys). Second, we seek to corroborate different indicators (e.g., a decrease in HIV prevalence, a decrease in STI, a decrease in multiple partners) in the same data set and across data sets. Third, we attempt to corroborate and further our understanding of the trends in the indicators with other information, including qualitative studies, expert opinion and programmatic data, for example. If the same conclusions are reached across the diverse data sources, different indicators and mix of methods, we have successfully "triangulated" a finding in the social science sense of using multiple data sources to externally validate a finding. If we the data sources lead in different directions, then we attempt to understand why, for instance, by examining for modifying variables, different sampling methods or different study quality.
We next formulate possible hypotheses that explain the data and can potentially answer the original key question, with particular attention to data trends that we have collected and reviewed. This process is inductive and empirical; we seek to identify hypotheses that explain all the data at hand and suggest a causal relationship between predictor variables (e.g., a public health intervention) and outcomes (e.g., risk behaviours or infection). It is very important to not discard data that do not fit a hypothesis. All data have to be accounted for or explained, and, while often a perfectly reasonable explanation is that an outlier was derived from a methodologically flawed study, outliers need to be examined closely before being excluded on methodological grounds alone. If an explanatory hypothesis fits all the data, it is confirmed; if not, we may need to gather additional data or reformulate the hypothesis. Epidemiologic criteria, such as those proposed by Hill , for assessing if the relationship between predictor and outcome variables is potentially causal can be especially helpful to assist in formulating hypotheses.
Triangulation is an iterative process. Refining hypotheses to better match the data is central to public health triangulation. We are searching for the hypothesis or explanation that is consistent with the preponderance of the data and is biologically or socially plausible. If the weight of the evidence is inconsistent with the hypothesis, the hypothesis should be rejected. It is important to mention that the findings from this data review process are not as dependent on statistical testing as in traditional experimental or observational studies. Triangulation results are based on repetition of findings from multiple data sources, often using findings derived from non-identical methodologies.
Accepting an hypothesis as the best answer to the question that was originally asked is based on the preponderance and strength of the evidence. When comparing alternative hypotheses, we ask:
Which hypotheses explain all or most of the data? Which are supported by the most independent sources of data?
Which hypotheses are supported by the most rigorous data, the best study designs and the most generalizable results?
Which hypotheses meet the most epidemiologic criteria for causality, especially biological or sociological plausibility?
Which hypotheses are supported by both the 'numbers' and the 'stories' (quantitative, qualitative and expert opinion)?
Would the likely biases, limitations and potential confounders in one or more studies in the analysis change conclusions?
Have all the alternative explanations been considered and rejected?
For example, in Mozambique prevalence of HIV in women attending antenatal clinics is substantially lower in the northern three provinces than in the rest of the country . Behavioral data from prior surveys indicate that sexual risk is just as high as in the rest of the country and condom use, if anything, is lower. However, male circumcision rates are far higher than in the rest of the country approaching 80% or more. Thus, we can suggest that male circumcision, which has been shown to be effective in three randomised controlled trials in other countries of the region, may be causally associated with the low prevalence. The public health action is to suggest male circumcision programs for other higher prevalence parts of the country and to evaluate its impact.
Stated another way, triangulation attempts to use the preponderance of the evidence collected to support the hypotheses, with particular attention paid to trends and consideration given to both supporting and refuting findings. Triangulation can be thought of like detective work, each new clue or data point can serve to confirm or refute your working hypothesis, or to lead you in search of additional data or alternative explanations. There are often multiple ways to arrive at the answer. Just like a good detective, a good practitioner of public health triangulation develops experience over time, which can allow him or her to develop narrower and better hypotheses, and know where to look for the best 'clues'. Often the best clues come from stakeholders who know the situation best; finding the right answer to questions is empirical, and initial ideas that lead to hypotheses are frequently best informed by stakeholders with an experiential sense of the social dynamics of HIV transmission in local populations.
Limitations of public health triangulation
Public health triangulation has several limitations. First are the data on which it is based. Data that directly answer the question being asked are not likely to be found, and inferences have to be made based on available data. These are often problematic: How free of bias are they? How likely are they to be externally valid? We have found that identifying and examining data that bear on different points in the chain of events that leads to the outcome in question is useful for strengthening the likelihood that the individual trend in question, such as declining HIV prevalence among women attending antenatal clinics in Uganda, is part of a larger series of events that has culminated in the observed trend.
Secondly we risk ecological fallacy. We often are faced with the situation in which we have trend data for predictor variables and trend data for outcome variables, but the predictor and outcome variables are not linked at the level of the individual survey participant. Thus, our interpretations may be the consequence of ecological fallacy if we are not able to refer to an underlying model that lays out how predictor variables are associated with outcome variables. We have found applying Hill's criteria  useful in aiding our understanding of the potential causal relationships between trends in predictor variables and trends in outcome variables. In fact, our framework for public health triangulation echoes one of Hill's criteria, consistency of results in several studies of different design.
Thirdly we face the problem of "data dredging"; that is, analyzing large data sets without null hypotheses and often failing to correct statistical tests for multiple comparisons. To guard against this, our practice is to be scrupulously aware of all the data at hand and only reject non-supportive data with great reluctance and for clear reasons. Additionally, the recommendation to conduct all phases of triangulation with the body of stakeholders is in part to prevent the preconceptions of one or a few persons from dominating the process.
Fourthly, we are concerned about the reproducibility of results. Different investigators should be able to come up with the same answer to a question that has been posed. We believe that reproducibility lies in a careful search for available data, careful scrutiny of the data themselves and neither over interpreting the data at hand nor ignoring data that do not fit hypotheses.
Public health triangulation is a process for the synthetic interpretation of data sets that either cannot be matched at the individual patient level or combined in meta-analysis; it can be a helpful adjunct to surveillance, to formal public health intervention research and to monitoring and evaluation (for examples, see [37, 38]) While many public health practitioners instinctively compare data from different sources to ascertain whether observed trends are consistent biologically and sociologically and mutually reinforce each others' external validity, we believe that a more formal examination of all available data sets, which often do not include those routinely examined by national AIDS control programs, such as vital statistics and data from other sectors like the military and police, can enhance the understanding of what is happening epidemiologically in a country. More usefully perhaps, the process of public health triangulation can also examine the impact of interventions, such as in Uganda, and suggest geographical areas and populations in need of interventions. This is perhaps public health triangulation's greatest utility.
Public health triangulation should not displace formal evaluation of interventions, carefully constructed surveillance systems or formal monitoring and evaluation activities, but it does offer an opportunity to compare and contrast the data generated by these activities with the end of improving public health outcomes. We believe that public health triangulation can be of assistance to the practitioner who needs to make public health decisions - allocating resources, initiating control programs, deciding to expand or discontinue specific prevention activities - and to evaluate their impact in real time.
List of Abbreviations
behavioral surveillance system
Demographic and Health Survey plus HIV testing
human immunodeficiency virus
Rapid Assessment and Response
Special United National Programme on HIV/AIDS
World Health Organization
We thank our colleagues in Botswana, Croatia, Malawi, Mozambique, Namibia and Tanzania for their contributions to the development and refinement of public health triangulation.
This work was supported by a cooperative agreement with the Global AIDS Program, Centers for Disease Control and Prevention, through the University Technical Assistance Project #U62/CCU922423. The findings and conclusions in this paper are those of the authors and do not necessarily represent those of the Centers for Disease Control and Prevention.
- Denzin NK, ed: Sociological Methods. A Sourcebook. 1970, Chicago: Aldine Publishing, 471-475.Google Scholar
- Donoghue TA, Punch KF: Qualitative Educational Research in Action: Doing and Reflecting. 2003, London: Routlege FalmerGoogle Scholar
- Cohen L, Manion L: Research Methods in Education. 1986, London: Croom HelmGoogle Scholar
- Webb EJ, Campbell DT, Schwartz RD, Sechrest L: Unobtrusive Measures: Nonreactive Measures in the Social Sciences. 1966, Chicago: Rand McNallyGoogle Scholar
- Denzin NK: The Research Act in Sociology. 1970, Chicago: Aldine PublishingGoogle Scholar
- Williamson GR: Illustrating triangulation in mixed-methods nursing research. Nurse Res. 2005, 12 (4): 7-18.View ArticleGoogle Scholar
- Halcomb E, Andrew S: Triangulation as a method for contemporary nursing research. Nurse Res. 2005, 13: 71-82.View ArticlePubMedGoogle Scholar
- Docherty B: Using triangulation in health services research. Prof Nurse. 2000, 16: 926-927.PubMedGoogle Scholar
- Arksey H, O'Malley L: Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005, 8: 19-32. 10.1080/1364557032000119616.View ArticleGoogle Scholar
- The Global Evidence Mapping Initiative: putting research into context. Accessed June 1, 2010., [http://www.evidencemap.org/]
- Pawon R: Evidence-Based Policy: a Realist Perspective. 2006, London: Sage PublishingView ArticleGoogle Scholar
- Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review-a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005, 10 (Suppl 1): 21-34. 10.1258/1355819054308530.View ArticlePubMedGoogle Scholar
- Cook DJ, Mulrow CD, Haynes RB: Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med. 1997, 126: 376-380.View ArticlePubMedGoogle Scholar
- Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A, Harvey J, Hsu R, Katbamna S, Olsen R, Smith L, Riley R, Sutton AJ: Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006, 6: 35-10.1186/1471-2288-6-35.View ArticlePubMedPubMed CentralGoogle Scholar
- Razum O, Gerhardus A: Methodological triangulation in public health research - advancement or mirage?. Trop Med Int Health. 1999, 4: 243-244. 10.1046/j.1365-3156.1999.00399.x.View ArticlePubMedGoogle Scholar
- Stoneburner R, Montagu D, Pervilhac C, Fidzani B, Gill W, Kennedy G, Spindler H, Rutherford G: Declines in adult HIV mortality in Botswana, 2003-2005: evidence for an impact of antiretroviral therapy programs [Abstract THLB 0507]. Abstracts of the XVI International AIDS Conference, Toronto, Canada. 2006Google Scholar
- Stoneburner RL, Low-Beer D, Tembo GS, Mertens TE, Asiimwe-Okiror G: Human immunodeficiency virus infection dynamics in East Africa deduced from surveillance. Am J Epidemiol. 1996, 12: 435-449.Google Scholar
- Low-Beer D, Stoneburner RL: Rapid communication change in reducing HIV: is Uganda unique?. Afr J AIDS Res. 2003, 2: 9-21.View ArticlePubMedGoogle Scholar
- Stoneburner RL, Low-Beer D: Population-level HIV declines and behavioral risk avoidance in Uganda. Science. 2004, 304: 714-8. 10.1126/science.1093166.View ArticlePubMedGoogle Scholar
- Stoneburner RL, Low-Beer D: Elements of sexual behaviour change associated with HIV prevalence declines in Uganda: comparative analyses of HIV and behavioural data in Uganda, Kenya, Malawi and Zambia [Abstract ThOrC721]. Abstracts of the XIII International Conference on AIDS, Durban, South Africa. 2000Google Scholar
- Joint United Nations Programme on HIV/AIDS: AIDS Epidemic Update: 2004. 2004, Geneva, Switzerland: UNAIDSGoogle Scholar
- Joint United Nations Programme on HIV/AIDS; AIDS Division, Ministry of Public Health, Thailand: Evaluation of 100% Condom Programme in Thailand (UNAIDS/00.18E). 2000, Geneva, Switzerland: UNAIDSGoogle Scholar
- Valdiserri R, Ogden L, McCray E: Accomplishments in HIV prevention science: implications for stemming the epidemic. Nat Med. 2003, 91: 881-886. 10.1038/nm0703-881.View ArticleGoogle Scholar
- Stoneburner RL, Des Jarlais DC, Benezra D, Gorelkin L, Sotheran JL, Friedman SR, Schultz S, Marmor M, Mildvan D, Maslansky R: A larger spectrum of severe HIV-1-related disease in intravenous drug users in New York City. Science. 1988, 242: 916-9. 10.1126/science.3187532.View ArticlePubMedGoogle Scholar
- Truong HM, Kellogg T, Klausner JD, Katz MH, Dilley J, Knapper K, Chen S, Prabhu R, Grant RM, Louie B, McFarland W: Increases in sexually transmitted infections and sexual risk behaviour without a concurrent increase in HIV incidence among men who have sex with men in San Francisco: a suggestion of HIV serosorting?. Sex Transm Infect. 2006, 82: 461-6. 10.1136/sti.2006.019950.View ArticlePubMedPubMed CentralGoogle Scholar
- The Global Fund to Fight AIDS, Tuberculosis and Malaria: Technical Background Document on the Scale and Scope of the Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria. 2006, Geneva, Switzerland: Global Fund to Fight AIDS, Tuberculosis and Malaria, Accessed 8 November 2008., [http://www.theglobalfund.org/en/terg/evaluations/sa1/]Google Scholar
- Scheer S, Chu PL, Klausner JD, Katz MH, Schwarcz SK: Effect of highly active antiretroviral therapy on diagnoses of sexually transmitted diseases in people with AIDS. Lancet. 2001, 357: 432-435. 10.1016/S0140-6736(00)04007-1.View ArticlePubMedGoogle Scholar
- World Health Organization, Department of HIV/AIDS, Department of Child and Adolescent Health and Development: Rapid assessment and response technical guide. 2003, Geneva, Switzerland: World Health Organization, (WHO/HIV 2002.22).Google Scholar
- Policy Project: A-squared: analysis & advocacy. [http://www.policyproject.com/abstract.cfm/2462]
- Whitson D, Horth R, Gonçalves S: Analysis of HIV prevention and modes of HIV transmission: Mozambique country synthesis. Analaysis of HIV prevention response and modes of HIV transmission: the UNAIDS-GAMET supported synthesis process. Joint United Nations Programme on HIV/AIDS, Regional Support Team for Eastern and Southern Africa, and World Bank-GAMET. Edited by: Calvin M, Gorgens-Albino M, Kaesdale S. Accessed 27 July 2010., [http://www.unaidsrstesa.org/files/MoT_0.pdf]
- Joint United Nations Programme on HIV/AIDS: UNAIDS Annual Report: Knowing Your Epidemic (UNAIDS/08.21E). 2008, Geneva: UNAIDSGoogle Scholar
- Altman DG: Systematic reviews of evaluations of prognostic variables. BMJ. 2001, 323: 224-228. 10.1136/bmj.323.7306.224.View ArticlePubMedPubMed CentralGoogle Scholar
- Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ: Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.View ArticlePubMedGoogle Scholar
- Huwiler-Müntener K, Jüni P, Junker C, Egger M: Quality reporting of randomized trials as a measure of methodologic quality. JAMA. 2002, 287: 2801-2804. 10.1001/jama.287.21.2801.View ArticlePubMedGoogle Scholar
- Hill AB: Environment and disease: association or causation?. Proc R Soc Med. 1965, 58: 295-300.PubMedPubMed CentralGoogle Scholar
- National AIDS Council, Republic of Mozambique: Report of Mozambique triangulation project: synthesis of data on trends in the national and local HIV epidemics and the reach and intensity of prevention efforts. Accessed June 2, 2010., [http://globalhealthsciences.ucsf.edu/PPHG/triangulation/country_reports.html]
- World Health Organization: Joint United Nations Programme on HIV/ADIS, Global Fund to Fight AIDS, Tuberculosis and Malaria. HIV Triangulation Resource Guide: Synthesis of Results from Multiple Data Sources for Evaluation and Decision-Making. 2009, Geneva, Switzerland: WHOGoogle Scholar
- University of California, San Francisco, Global Health Sciences, Prevention and Public Health Group: Triangulation: country reports. Accessed June 2, 2010., [http://globalhealthsciences.ucsf.edu/PPHG/triangulation/country_reports.html]
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/10/447/prepub