Table 1 Studies included in review: summary of data extracted

Bianchi

2018a

Nutrition /obesity

“to identify configurations of intervention characteristics associated with, and those not found to be associated with, statistically significant reductions in the demand for meat”

Crisp

Any (included studies all high income countries)

Any

Micro-environment interventions to reduce meat consumption

Published analyses (systematic review)

Quant

Intervention component

21 out of total of 22 interventions identified

Reduction in consumption, purchase, or selection of meat

Y

Y

N

N

N

Y

Y

Bianchi

2018b

Nutrition/obesity

“to identify combinations of intervention characteristics associated with significant reductions in the demand for meat”

Crisp

Any

Any

Interventions targeting conscious determinants of human behaviour to reduce the demand for meat.

Published analyses (systematic review)

Quant

Intervention component

31 out of a total of 59 interventions identified

Reductions in actual or intended meat consumption

Y

Y

N

N

N

N

Y

Blackman

2011

Health inequalities

“to find configurations of these conditions with either the narrowing or not narrowing outcomes.”; “enables causal arguments to be made rigorously by creating a very close correspondence between theory and data analysis, analysing evidence in ways that directly address theoretical arguments about what matters to achieve some outcome: how it happens. This makes QCA especially appropriate for policy research; the process of defining conditions is then also a process of identifying conditions necessary for a policy outcome.”

Crisp

UK

Those living in ‘deprived’ areas/regions, and at-risk of premature death from cancer and cardiovascular disease (CVD)

Target setting to reduce inequalities (Local policies and service delivery to identified disadvantaged areas)

Primary data (generated for study)

Secondary data

Quant

Region/ subnational area - England’s Spearhead areas

27 of possible 70 (27 complete questionairres returned for both cancer and CVD)

Not/narrowing cancer gap; or not/narrowing CVD gap

Y

Y

N

N

Y

Y

N

Blackman

2010

Health inequalities

“QCA […] assumes that outcomes are produced by variables acting together in combination, including the possibility of different combinations leading to the same outcome”; “using fsQCA to explore how the attributes configured against each outcome.”

Crisp

UK - North-West England

Those living in ‘deprived’ areas/regions, and at-risk of premature death from cancer and circulatory diseases

Local policies and service delivery to identified disadvantaged areas

Primary data (generated for study) Secondary data

Qual and Quant

Region/ subnational area - local strategic partnerships (LSP)

15 local authority areas in North West England of 21 LSP areas in the North West region that were among the most deprived 88 local authority areas in England (14 and 15 complete questionairries were returned for circulatory diseases and cancers, respectively)

Narrowing OR widening cancer gap; or narrowing OR widening circulatory diseases gap

Y

Y

N

N

Y

Y

N

Blackman

2013

Health inequalities

“enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes: in this instance, whether inequalities in conception rates do or do not narrow, compared with the England average.”

Crisp

England, deprived local authority areas

Women under 18

Local policies and service delivery addressing pregnancy in those aged under-18

Primary data (generated for study) Secondary data

Quant

Region/ subnational area – local authority

27 from 70 local authority areas

A narrowing teenage pregnancy rate gap compared to the national average OR a gap that had not been narrowing.

Y

Y

Y

N

Y

Y

Y

Bruer

2018

Mental health

Goal to “to evaluate outcomes and processes simultaneously … [QCA] holds promise as a way to do this”

Fuzzy

Nepal, West Chitwan

Primary care patients

Programme for Improving Mental Healthcare’ (PRIME) Mental Health Care Plans (MHCPs)

Secondary data

Quant

Organisation - health facilities

10 of the 10 health facilities in Chitwan where the PRIME MHCPs were implemented

Effectiveness of the intervention:increased service utilisation

Y

Y

Y

Y

Y

Y

Y

Brunton

2014

Community engagement

“Given that interventions targeting social determinants of health are necessarily complex (Medical Research Council 2008), this method [QCA] is well-suited to examine the components of effective interventions in maternity and early years interventions.”

Fuzzy

OECD countries (USA, UK, RoI)

Disadvantaged pregnant women and new mothers

Community engagement – orientated to antenatal care, breastfeeding, child illness prevention and parenting

Published analysis (systematic review)

Quant

Intervention

24 of 29 studies (a sub-set of the 131 trials included in the braoder review of studies)

Effectiveness of the intervention.

N

Y

Y

Y

Y

Y

Y

Burchett

2018

Nutrition/ obesity

QCA method used to attend to “the inherent variance in intervention content, context and outcomes […] to explore the critical features of WMPs [weight management programs] for children to understand the mechanisms through which interventions have the impact that they do.”

Crisp

UK

Children and Young People

Lifestyle weight management interventions

Published analysis

Quant

Intervention

15 (most/least effective) of 30 interventions

Intervention effectiveness

N

Y

N

N

N

Y

N

Chippone

2018

Nutrition/ obesity & PA

“to determine necessary and sufficient technical assistance conditions supporting NAP SACC [Nutrition and Physical Activity Self-Assessment in Child Care] outcomes.”; “to account for equifinality”

Fuzzy

USA

Children and Young People

Technical assistance

Primary data (generated for study)

Quant

Intervention (‘ECE programs’)

15 highest performing programs and 15 lowest performing programs (from 10 collaboratives, comprising 84 early care and education programs)

Effectiveness of technical assistance

Y

Y

N

N

Y

N

N

Fernald

2018

Chronic condition management

“to identify conditions/sets important for successful implementation. Qualitative Comparative Analysis (QCA) maintains complexity in the analytical process and allows for multiple pathways to achieve the outcome”

Fuzzy

USA

Primary care patients

Community-Created Self-Management Support (SMS) Tools in Primary Care Practices

Primary data (generated for study)

Qual

Organisation (primary care practice)

16 practices of 16 enrolled in study

Routine SMS implementation in primary care setting

Y

Y

Y

N

Y

Y

Y

Ford

2005

General population health status

“allows for the exploration of the necessity, sufficiency and interactions among the three core public health functions and relates them to the outcome of interest—health impact assessments.”

Crisp

USA

Whole population

Assessment, assurance and policy development functions (Adherence to IoM’s recommendations)

Secondary data

Quant

Region/ subnational area: State

41 of 50 states

Population health improvement 1990–2000 (composite measure based on United Health Group’s ratings) above or below average

N

Y

N

N

Y

Y

Y

Glatman-Freedman

2010

Vaccine adoption and implementation

“to examine the alternative combinations of factors that are conducive to the success or failure of new vaccine introduction”

Crisp

African countries eligible for GAVI funding

Organisations

GAVI funding

Secondary data

Quant

Country

35 (all eligible countries)

Country is in one of 3 groups: with both Hep B and Hib vaccines introduced; just hep b introduced; or neither introduced

N

Y

N

N

N

Y

Y

Harris

2019

Chronic condition management

“to identify the combination of intervention components and processes that are aligned with successful intervention implementation”

Fuzzy

Any (“Most studies were conducted in North America in socially disadvantaged populations.”)

Children and Young People

Asthma management interventions (“if their purpose was to help children improve management of their asthma by increasing knowledge, enhancing skills, or changing behaviour.”)

Published analysis (systematic review)

Qual and Quant

Intervention

33 of 55 studies

Implementation of intervention successful

Y

Y

Y

Y

Y

Y

Y

Harting

2019

Programme implementation

“Our theoretical framework indicates that it is the combination of conditions that is important for network performance, rather than the influence of conditions separately. Therefore, we performed a fuzzy set qualitative comparative analysis (fsQCA)”

Fuzzy

Netherlands, municipalities

Policy actors

Multi-sectoral network

Gezonde Slagkracht (Decisive Action for Health) program which provided resources for multisectoral networks for action on overweight, alcohol and drug abuse and/or smoking

Primary data (generated for study)

Quant

Public Health Policy Networks (in municipalities or collaborations of municipalities)

29 of the 34 Dutch public health policy networks in the programme

Network performance - indicated by implementation of more than 3 non-educational PH strategies

N

Y

Y

Y

Y

Y

Y

Hartman-Boyce

2018

Nutrition /obesity

“to identify combinations of intervention components associatedwith statistically significant changes (P < 0.05) in the desired direction for at least one of the foods targeted by the intervention […] to augment the narrative review.”

Crisp

Any

Food store consumers

Food purchasing interventions

Published analyses (systematic review)

Quant

Intervention

89 interventions, reported in 35 studies

Sucessful implementation of intervention (“statistically significant changes in the desired direction for ≥1 of the foods targeted by the intervention”)

N

Y

N

N

N

Y

Y

Kahwati

2011

Nutrition /obesity & PA

“to identify facility conditions or combinations of conditions associated with larger 6-month patient weight-loss outcomes. QCA is a method that allows for systematic cross-case comparison to better understand causal complexity.”

Crisp

USA

Adults (Veterans exposed to the MOVE! Weight Management Program)

MOVE! Weight Management Program for Veterans

Primay data and secondary data

Qual and Quant

Organisation - health facilities

Twenty-two facilities of 239 facilities eligible for selection (“Eleven sites with larger outcomes and 11 sites with smaller outcomes”)

Intervention effectiveness (“larger patient weight-loss outcomes”)

Y

Y

Y

N

Y

Y

Y

Kane

2017

Programme implementation

“To determine which combination of elements of capacity is most prevalent among the awardees that successfully implemented objectives”; “Programs such as CCPW [Communities Putting Prevention to Work] invove multiple components that may work together and different program models can lead to success. This method accommodates the complexity associated with evaluating such programs.”

Crisp

USA

Organisations

Communities Putting Prevention to Work (CPPW) program

Primary data (generated for study)

Qual

Organisation - awardee (“In most cases, the lead organization for an awardee was a city or county health department, although community-based organizations served as leads for a few awardees”)

22 of 50 community awardees

Intervention implementation effectivesness (“completion of approximately 60% of work plan objectives”)

Y

Y

Y

N

Y

Y

Y

Kien

2018

PA

“We chose QCA as the method of analysis, as we investigated a small number of cases and aimed to answer a question related to the combinations of conditions and not related to the identification of the independent influence of a variable. Furthermore, we were interested, in whether different combinations of causal conditions could lead to the same outcome.”

Fuzzy

Austria

Children and Young People

School based health promotion programme: “Classes in Motion”

Primary data (generated for study)

Qual and Quant

Classes

24 classes out of 26 classes that received the intervention

Percent of children showing an improvement in emotional and social school experience.

Y

Y

Y

Y

Y

Y

Y

Lubold

2017

Breastfeeding

“to examine the combinations of conditions leading to both high and low national breastfeeding initiation”

Fuzzy

OECD countries

Mothers

Baby-Friendly Hospital Initiative (BFHI)

Secondary data

Quant

Country

18 countries of 34 OECD countries

Percent of breastfeeding initiation

N

Y

N

Y

Y

Y

Y

Lucidarme

2016

Programme implementation

QCA as most appropriate method because “we have a combination of a relatively large number of determinants and a small number of cases.”; “a large amount of in-depth qualitative data was collected and CCMs [Configurational comparative methods] are able to deal with such large amounts of qualitative data.”

Crisp

Flanders, Belgium

Organisations

Local health promotion networks (LHP)

Primary data (generated for study)

Secondary data

Qual and Quant

Network

13 of 13 LHPs

Composite score of effectiveness of network from 5 measures at Community level (Awareness and change in awareness of ‘10,000 steps’ programme) and Network level (measures of participation, and actions at municipal and regional level)

Y

Y

Y

Y

Y

Y

Y

McGowan

2019

Mental health & Community Engagement

“QCA is therefore of potential use in the evaluation of the effectiveness of complex public health interventions as applied to small populations”

Crisp

UK, ‘Big Local’ (BL) areas (disadvantaged areas in UK)

Adult participants in BL partnerships (residents, workers, volunteers in locality)

A community-led empowerent initiative

Primary data (generated for study)

Quant

Person /individual

48 participants of 65 participating in a Wave 2 survey

Improvement in mental health

N

Y

Y

N

Y

N

N

Melendez-Torres

2018

Nutrition /obesity & PA

“we aimed to understand why some interventions appeared to work better than others, or, whether specific combinations of WMP [weight management program] features were associated with greater intervention effectiveness. [...] QCA was particularly suitable [...] as it facilitates the identification of configurations of various intervention and other contextual features that are (or are not) present when the intervention has been evaluated and found successful (or not) in obtaining a desired outcome”

Fuzzy

Any (but notes in original review that over half of included studies were from US)

Overweight or obese adults

Weight management programs

Published analysis

Quant

Intervention

20 (10 most effective; 10 least effective) from 40 intervention arms within 30 trials

Pathways to high (and low) intervention effectivess

N

Y

Y

Y

Y

Y

Y

Parrott

2018

Nutrition/obesity

“QCA is much more suited to answering the question: For whom and under what conditions does the intervention make a difference”

Crisp

Any

Children and Young People

Pediatric weight management (PWM) interventions

Published analysis (systematic review)

Quant

Intervention component

209 separate treatment arms from 99 controlled trials, were included in this analysis.

Positive weight status outcomes

N

Y

Y

N

Y

Y

Y

Roberts

2018

Vaccine adoption and implementation

“Qualitative comparative analysis (QCA) is a formalized qualitative analytic approach that can be leveraged to determine which sets of state policies may be necessary or sufficient for high state-level HPV vaccination uptake. States have enacted multiple, often overlapping policies that may influence HPV vaccine uptake; QCA is well suited for characterizing these complex relationships.”

Fuzzy

USA

Children and Young People

State policies

Secondary data

Quant

Region/subnational area

51 (of 50 States + Washington DC)

Higher levels of HPV vaccine uptake in adolescent boys and girls

N

Y

Y

N

Y

Y

Y

Thomas

2014

Community engagement & Breastfeeding

“the context for this paper: a need to identify important components of interventions when making commissioning decisions, but a lack of established methods of synthesis which enable such investigations. We therefore examine an analytical technique, [QCA] which has been designed to overcome some of the limitations outlined above.”

Fuzzy

OECD countries

Expectant and new mothers

Community engagement programmes directed toward expectant and new mothers to promote breastfeeding.

Published analysis (systematic review)

Quant

Intervention

12 studies as a subset of a review that included 319 studies

Intervention effectiveness (“membership in the set of highly effective interventions”)

N

Y

Y

Y

Y

Y

Y

Warren

2013, 2014

Chronic condition management

To capture complexity (particularly on how policy interventions work across heterogeneous contexts) (2013); “QCA addresses multiple causation” (2014)

Crisp

UK - North-East England

Adult working age recipients of incapacity benefit (IB)

Case management to help benefit recipients return to work

Primary data

Quant

Individual

131 participants receiving the intervention

Self-reported health (EQ. 5-S score) improves/does not improve relative to UK population

Y

Y

N

Y

Y

Y

Y